2

Please Note: I'm not asking what to do here. I'm asking why this is happening, in order to understand Google Search Console's behaviour.

So this has been driving me nuts for weeks. I've made repeated removal requests in Search Console for 404 error pages that Google somehow sees as coming from pages long since gone, including a Yoast WP SEO XML sitemap (I even removed the Yoast plugin months ago). I've been repeatedly requesting removal of all these pages for months now, but they keep popping up as sources for my 404's.

I've run Screaming Frog, Deepcrawl, combed though the site and Google's indexes, and not seeing a trace of these pages anywhere but from inside Search Console...

Search Console Mystery 404s

  • Just ignore them if you're in hurry. There are so many webmaster use search console and Google need to call many of API to update the result, so it take some time. If you want updated result then try to click on marked as fixed, so may be google will don't include those links again. – Goyllo Dec 30 '16 at 10:48
  • 1
    Search Console is often way behind the curve. You will see errors and notices for things that once existed but no longer does. Do not worry about what you see. As well, NEVER(!), NEVER(!), NEVER(!) Mark a 404 error as Fixed if the page in question does not exist. – closetnoc Dec 30 '16 at 15:05
  • @Goyllo I've been requesting removal and marking as fixed for months. They keep coming back. – Mattypants Dec 30 '16 at 16:29
  • @closetnoc Why not mark a 404 error as Fixed if the page in question does not exist? – Mattypants Dec 30 '16 at 16:30
  • If a page correctly does not exist, then the 404 error is correct. Marking a 404 error as Fixed tells Google that the page should exist and to try again. It is a never ending loop. You tell Google the page should exist, Google tries again, the 404 remains as it should, and you tell Google the page should exist, Google tries again... and so on, and so on. Only Mark a 404 as Fixed when a page should exist and you want Google to find it. – closetnoc Dec 30 '16 at 16:50
2

It seems all the pages from a previous version of the site - all with php extension'd pages - continously come back as 404, including the old XML sitemaps from the long-ago-removed Yoast SEO plugin.

To encourage Google to forget about these long gone and never coming back URLs or to at least help differentiate them from more current 404s, you could try returning a "410 Gone" instead of the usual "404 Not Found". This is a stronger signal to Google that these URLs are not coming back.

You could implement this using Apache mod_rewrite in either .htaccess or your server config. For example, to respond with a "410 Gone" for all .php URL requests, then you can do something like:

RewriteEngine On
RewriteRule .+\.php$ - [G]

Although, when you say "previous version", it's not simply a previous version of the same site? I assume the content is no longer relevant - so a redirect would indeed be inappropriate.

  • For the most part, the 404s are from a previous version of the same site. But some stray, irrelevant files, including/especially the old Yoast sitemaps I'll use a 410 Gone on those specific files. Awesome, thank you. – Mattypants Jan 2 '17 at 21:10
  • 1
    I can just do them one at a time like this: redirect gone /path/page.html – Mattypants Jan 2 '17 at 21:18
  • Yes, you can do that. (Just be wary of any potential conflicts with mod_rewrite, if you are already using mod_rewrite for other rewrites/redirects.) – MrWhite Jan 2 '17 at 21:30
  • I have another issue with mod_rewrite, but doubt it'll clash here – Mattypants Jan 2 '17 at 21:42
1

To remove the 404 pages from the search console, redirect those 404 pages to the relevant pages so that Users who land on those pages can be redirected to proper pages and doesn't affect User Experience.

After that, you can mark as fixed and it won't be coming back to you again.

  • 3
    But only redirect the user if there is a "relevant" page to redirect to. If there is no relevant page then don't redirect. The 404 page should then provide the best information to keep the user engaged. – MrWhite Jan 2 '17 at 19:07
  • 1
    Yes, exactly. I totally agree with you. – Shivbhadrasinh Gohil Jan 6 '17 at 12:26
  • Just thought it was worth emphasising since we seem to encounter a lot of users/developers that choose to redirect simply to avoid the 404 and do things like mass redirecting to the home page etc. (which is generally bad). – MrWhite Jan 6 '17 at 12:33
1

There is really no cause to become nervous. Your GSC displays 7 404 errors. All of them are old and/or non-existing.

Remember, GSC gets data from many data centers, there are many latencies on the way. Every some months GSC applyes any data update and looses some data (like not long ago, Google confirmed loosing of up to 50% of link data for some sites).

Well, then no panic and happy new year.

  • I'm not panicking, I'm trying to get to the bottom of the reoccurring 404's. There are only seven 404 errors because I'm continuously requesting removals and marking as fixed. The continuously return as 404. – Mattypants Dec 31 '16 at 14:45
  • 1
    "...and marking as fixed" - as @closetnoc already stated in comments, these URLs should not be marked as "fixed" if they still return a 404. If they still return a 404 then they are not "fixed". The "mark as fixed" option is for when you genuinely had a mistake/config error on your site - when a page that should have existed, but for some reason was returning a 404 instead. Once you correct the error and the URL now returns valid content and a 200 OK response then you can mark it as fixed. But most 404s, like the ones you mention, are not fixable. – MrWhite Dec 31 '16 at 15:45
  • @w3dk Quite annoying to not have the option at least to remove them from view, at least to keep my errors list organised. – Mattypants Jan 2 '17 at 18:55
  • @Mattypants Yes, it can be annoying. But as mentioned, if you only have a few then you can just ignore them. However, it is particularly frustrating when you have 100s or even 1000s of legitimate 404s as it then becomes near impossible to check for any new 404s that might be the result of some kind of config error, that need to be fixed. – MrWhite Jan 2 '17 at 19:12
  • 1
    @w3dk Which brings me back to the original problem. It seems all the pages from a previous version of the site - all with php extension'd pages - continously come back as 404, including the old XML sitemaps from the long-ago-removed Yoast SEO plugin. The pages aren't anywhere on the site, and searching for the pages in Google results in no hits, which is good, but the returning 404 errors are annoying, because I like to resolve questions marks. The pages are nowhere. I'd had them 301'd to the new Wordpress pages for a few months until I removed them, and now seeing the 404's again. – Mattypants Jan 2 '17 at 19:55
-3

Just ignore these urls, Google takes a long time to update there queries, so this is not any serious issue.

  • This answer doesn't say anything that hasn't already been said here. – Stephen Ostermiller Feb 16 at 13:04

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.