2

The site I am working on has around 800 products, some of these can have up to 2000 PDF and CAD drawings each.

The links to these PDF and CAD files are combined into a single object and then linked of a product page with the url structure of:

$domain/downloads/download/product/$combination_id

As there are so many drawings we have created a system that will completely delete the combinations and re-generate them (making new combination_id's in the process)

I have asked Google to not index these URLs and they don't appear in organic search results.

However Google Webmaster Tools has recently indexed the site and has thrown up a huge number of 404s for the old combination ids that have been deleted.

My question is, should I:

  1. Block these urls with a /downloads/download/product robots.txt rule?

  2. Create 301 redirects for each old combination that would then have to be updated whenever a new version of a drawing is added?

  3. "Slugify" the combination_id's so they don't change when the drawings are updated and create 301 redirects for all the existing combinations.

  • A URL slug is the portion of the URL that contains words describing the document. It is usually in addition to a numeric ID. In /9393.pdf vs /9393-widget-guide.pdf the URL slug would be "widget-guide", but the URL would still have an ID in it. I think it would be clearer if you said "remove the ID from the URL" rather than "slugify". – Stephen Ostermiller Jun 13 '18 at 11:17
1

By the sounds of it, this is really just one (of many) page that is being updated (with PDFs and CAD files), rather than an entirely new page - although that would seem to be how you have currently implemented this.

Consequently I think the URL should remain consistent and not change. So, this relates to your option #3: "Slugify the combination_id's so they don't change". You will need to implement 301 redirects for all "old" URLs (combination IDs).

If you blocked the old URLs with robots.txt then the URLs themselves could still remained indexed for an extended period of time, as the redirect will not be seen. Unless of course your intention is to block all product URLs (but I don't think you are really suggesting that)?

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.