Hot answers tagged web-crawlers
Check if the these pages are linked from any other page. Remove those links and mark the errors as fixed. Make sure there are no backlinks on that pages. If there are any backlinks coming to that page make sure you display 404 and submit all those domains to disavow file.
Only top voted, non community-wiki answers of a minimum length are eligible