The site I am working on has around 800 products, some of these can have up to 2000 PDF and CAD drawings each.
The links to these PDF and CAD files are combined into a single object and then linked of a product page with the url structure of:
$domain/downloads/download/product/$combination_id
As there are so many drawings we have created a system that will completely delete the combinations and re-generate them (making new combination_id's in the process)
I have asked Google to not index these URLs and they don't appear in organic search results.
However Google Webmaster Tools has recently indexed the site and has thrown up a huge number of 404s for the old combination ids that have been deleted.
My question is, should I:
Block these urls with a /downloads/download/product robots.txt rule?
Create 301 redirects for each old combination that would then have to be updated whenever a new version of a drawing is added?
"Slugify" the combination_id's so they don't change when the drawings are updated and create 301 redirects for all the existing combinations.
/9393.pdf
vs/9393-widget-guide.pdf
the URL slug would be "widget-guide", but the URL would still have an ID in it. I think it would be clearer if you said "remove the ID from the URL" rather than "slugify".