It's my understanding that URLs are of the format example.com/something/somethingelse.
It's my understanding that parameters follow the URL with a question mark example.com?l=fr_FR
My CMS website has language translations that use parameters. The example above is for the French language version of my site.
I would like to block all non english translations from Google Index using robots.txt.
In the blocked URLs tool in GWT I tried to test this:
# robots.txt generated at http://www.mcanerin.com
User-agent: *
Disallow:
Disallow: /cgi-bin/
Disallow: ?l=
Against the following URL, which appeared as one of the duplicate page titles in HTML improvements.
https://example.com/reports/view/884?l=eu
This is my first time fiddling with this tool in GWT so I'm not sure if I'm using it right.
The test results for Googlebot says
Allowed by line 3: Disallow:
I wanted the code to prevent Google indexing any URLs that contain the following string
?l=
Here are some examples of URLs I'd like to block from the index. These URLs generate duplicate titles according to GWT. /reports/view/884?l=km /reports/view/884?l=ne_NP /reports/view/884?l=te /index.php/page/index/12?l=fr_FR&l=hy_AM /index.php/page/index/12?l=ht_HT&l=bn_BD /index.php/page/index/12?l=hu_HU&l=hy_AM
Can I tell the robots to exclude URLs with tags that contain
?l=