1

I have been scratching my head about this for months, we have about 100 Not found links with 404 response codes.

The response code is correct as these pages do no exist anymore, however the thing that is confusing when i look to see which pages they are linked from they are calling from page which do not exist and have redirects on them or from the site map.

For example if the page "/red-boots" had a response code of 404. I check where it is linking from and it says /boots/shoes" and the sitemap.

When i check the url "/boots/shoes", it has a redirect to "/shoes" however google isnt seeing that redirect it is just finding the old links on "/boots/shoes".

I have regenerate a sitemap and that hasn't helped, i marked them all as fixed and within 2 days they were all back. So if somebody could shine a bit of light on why google is looking at unpublished pages and not following the redirects that would be brilliant.

Improve this question
4
  • 1
    Never mark legitimate 404 errors as being fixed!! You are telling Google that the pages should be found and to try again. For any page found to be 404, Google will retry that page a number of times for a period. If the page is actually gone, you simple let Google do it's thing and it will remove the URL providing there is not link to that page. Otherwise, marking a valid 404 page as being fixed restarts this process again from the beginning and the error will never be resolved until you leave it alone.
    – closetnoc
    Commented Apr 4, 2016 at 14:58
  • As well, do not put into your sitemap any URLs that do not exist or redirect. Sitemaps are not for URLs that exist, but for pages that exist.
    – closetnoc
    Commented Apr 4, 2016 at 14:59
  • I'd suggest running some sort of link scanner like Xenu or Screaming frog SEO spider. They might spot broken links for you.
    – Mike -- No longer here
    Commented Apr 4, 2016 at 21:23
  • We use screaming frog and that isn't finding any broken links, does that mean that google just hasn't updated in a while as the pages it isn't reading the redirects on are 301 redirects.
    – Adam Allen
    Commented Apr 5, 2016 at 9:18

2 Answers 2

Reset to default
3

There are two types of redirects :

1) 301 : permanent redirect -> If you have added this, then google will know that you have permanently moved this page and will store this in its index.

2) 302 : temporary redirect -> This will not let crawler index the redirection permanently.

Firstly make sure that the redirection is 301 permanent redirect. Also google takes some time(may be a week or even a month) to reflect these redirection changes. These outdated links/old behavior on pages will continue to exist for sometime and then eventually will leave the index.

Also if some of your pages are permanently discontinued, then its better to throw HTTP code 410 to let crawler know that you have removed this page from site and hence crawler should also remove it from index. In case of 404 it continues to hit that page as it see it like a temporary issue on site.

Do check the sources from where the old pages are still linked. It might be the case that you have updated the sitemap, but some of the old pages which you have not removed continue to have old links and hence crawler reach them. If you are using webmaster tool it will show you all details in linked from tab in errors section.

Improve this answer
0

I've seen this on several sites which have been around for a while. Links and URLs change over time, but Google doesn't tend to forget them. Sometimes the "Linked from" URL has been gone for years but Google still flags these up from time to time, probably just to make sure you don't miss anything.

As @abhinsit mentions, you can use a 410 header on anything that is absolutely gone for good. Google may still come back to check in future, but generally these are probably safe to ignore.

Improve this answer
2
  • It does seem a lot of the links that are in the "Linked from" section are old locations as we restructured our URL system about a year ago. Will these ever disappear?
    – Adam Allen
    Commented Apr 5, 2016 at 10:49
  • Not in my experience.
    – Andrew Lott
    Commented Apr 5, 2016 at 14:18

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.