The most recent versions of Joomla no longer block the /media/
and /templates/
folders:
User-agent: *
Disallow: /administrator/
Disallow: /bin/
Disallow: /cache/
Disallow: /cli/
Disallow: /components/
Disallow: /includes/
Disallow: /installation/
Disallow: /language/
Disallow: /layouts/
Disallow: /libraries/
Disallow: /logs/
Disallow: /modules/
Disallow: /plugins/
Disallow: /tmp/
Not all extensions stick to the guidelines of where to place CSS and JS files etc, so a good work around is to allow Google to access these files regardless of where they are found.
You can achieve this by inserting a few lines to the start of your robots.txt
file like this:
#Googlebot
User-agent: Googlebot
Allow: *.css
Allow: *.js
User-agent: *
Disallow: /administrator/
Disallow: /bin/
Disallow: /cache/
Disallow: /cli/
Disallow: /components/
Disallow: /includes/
Disallow: /installation/
Disallow: /language/
Disallow: /layouts/
Disallow: /libraries/
Disallow: /logs/
Disallow: /modules/
Disallow: /plugins/
Disallow: /tmp/