1

I have a online store with Magento, which means there are a lot of parameters to filter categories. Small example:

www.example.com/chats/distributeurs-fontaine/

www.example.com/chats/distributeurs-fontaine/?limit=12

Since it's bad for SEO, we did 2 things for the 2nd URL not to index: we blocked it via robots.txt and used a canonical.

Robots.txt line:
Disallow: /*?limit*

Canonical:
<link rel="canonical" href="http://www.example.com/chats/distributeurs-fontaine/" />

Still, the 2 URLs are indexed. I'll admit we didn't set proper URL parameters instructions in Webmaster tools from the start, but I would have thought we did enough.

Do you have an idea why the canonical didn't work? And do you see a potential solution to get the thousands of bad URLs out?

Improve this question
4
  • How long have your changes been in place for?
    – nathangiesbrecht
    Commented Mar 5, 2015 at 15:20
  • Robots.txt and canonical have been active since launching the website 1 month ago, URL parameters have been active for +/- 1 week. (I don't think modifying these parameters retroactively deletes URL though, am I wrong?)
    – user3730012
    Commented Mar 5, 2015 at 15:22
  • I would have thought the canonical would have dealt with that, but if the websites that new it may not be crawled often enough for Google to have noticed yet. In my experience setting the parameters properly in GWT does actually have a pretty good effect, over time. You should probably do that anyway, just as a "best practices" type of thing. FYI/FWIW - disallowing a URL in robots.txt only means that bots won't crawl it, doesn't mean they won't index it.
    – nathangiesbrecht
    Commented Mar 5, 2015 at 16:08
  • 1
    "the 2 URLs are indexed" - Are they indexed with a full description in the SERPs? Or does it reference "robots.txt"?
    – MrWhite
    Commented Mar 5, 2015 at 16:21

0

Reset to default

Browse other questions tagged or ask your own question.