2

Currently, in Google Search Console, we have some errors in "Not Found" for which page currently does not exist on our site and there is no 'Linked from' data. These pages will not come back in future. What should be I do with such type of URL?

  1. 410: Return "Gone" as a response code.
  2. 404: Continue to serve "Not Found."
  3. Stop indexing them using the remove URLs tool.

I want to stop indexing such type of URL as soon as possible. So from above three options, which should I pick?

I search on internet and got mixed answer, some say to ignore it and Google will stop indexing it after some time. Some say to return 410 as it will stop getting indexed faster. Some say use removal tool carefully (but mostly I found not to use removal tool). All the answers I've found are pretty old, so I'm not sure which advice to take.

Improve this question
2
  • 1
    "3. ... using Remove URLs Tools." - This isn't a separate option. The URL must already return a 404/410 status for the "removal tool" to have any lasting effect. This is just a method to help speed up the process, primarily if you need to remove sensitive information.
    – MrWhite
    May 24, 2018 at 16:38
  • If Google is showing a "Not Found" error in Google Search console, it will stop indexing the page within 24 hours if it hasn't already removed it from the search index. Google doesn't index error pages.
    – Stephen Ostermiller ♦
    Mar 22 at 14:48

3 Answers 3

Reset to default
2

If the pages existed at some point, and now have logical direct equivalents, redirect the request (with a 301) to the new URLs.

If the pages existed at some point, don't any more, don't have a direct equivalent and will never exist again; return a 410.

If the pages never existed, return a 404.

The search console tells you about these things in case they are caused by misconfigurations on your site. If you aren't linking to these pages and there aren't any inbound links to them, then there is no problem - Google will stop indexing these pages in time.

Improve this answer
7
  • I want to stop indexing ASAP, so should I return 410 or use some deindex(remove url tool)?
    – yajiv
    May 24, 2018 at 9:19
  • Use whichever status code is appropriate given the details in my answer. Why do you want to stop the indexing ASAP? Like I said, it likely isn't causing any problem.
    – Tim Fountain
    May 24, 2018 at 9:20
  • If the pages are appearing in Google search results, that's a different matter; and in that case you could use the URL removal tool as long as the URLs return a 404/410 status code.
    – Tim Fountain
    May 24, 2018 at 9:27
  • In deindexing we just have to give url, and it will stop indexing that url, So it seem easy than returning 410. and if we mark url 410 than also google tried it 2 to 3 times before stop indexing it.(I am saying this because in my webmaster search console, there are some url whose status code 410 but it still there in webmaster).
    – yajiv
    May 24, 2018 at 9:37
  • They won't permanently stop crawling a URL if they see a 410 just in case you were returning it by mistake. But again - it's perfectly fine for a bot to crawl a 4xx a few times before stopping.
    – Tim Fountain
    May 24, 2018 at 10:31
0

According to John Mueller from Google*, it might still come back and crawl the URLs when it has some free time. The only way I know of is to add the URL to robots.txt. Note the URL will not disappear from Google Search Console (it will appear under blocked by robots.txt), but it will not try to crawl it.

Improve this answer
-2

Your 3 primary options are 404, 410 and 301.

With 404: Google will likely continue to recrawl these pages in the future to see if they no longer return a 404 status. This will use up some of Googlebot's crawl resources that it might have used for other pages on your site, and it will keep these pages indexed until Google decides that the 404 is actually permanent. Any link juice that was pointing to this page will potentially be lost unless you return a 404 page that has links pointing to your home page and other pages, in which case some of the link juice will pass to the pages that you link to.

With 410: Google will immediately notice that the page is "gone". It should deindex the page immediately and stop recrawling that URL. All link juice pointing to that page will most likely be lost.

With 301: The URL redirects to your homepage. All link juice pointing to that page is passed to your home page, retaining as much domain authority for your site as possible. Google sees the 301 redirect as permanent and therefore should deindex that URL, and will most likely cease crawling it immediately.

In the past, I've preferred to 301 redirect these pages to the home page. It retains link juice and keeps Googlebot on your site. Since Googlebot will be redirected to the homepage, it will not only crawl that, but also potentially will crawl the links on your homepage that point to other pages on your site. And it also has the added benefit of informing Google that the page is permanently removed. 410 loses link juice and 404 isn't considered permanent.

Improve this answer
6
  • So what do you suggest what should I do? 301 or 410?(you have said in past you preferred 301, but what about in present.)(sorry for this stupid question)
    – yajiv
    May 24, 2018 at 5:32
  • I still use the 301. If it were my site that's how I'd prefer to handle it
    – Michael d
    May 24, 2018 at 6:04
  • 4
    One question, Isn't it bad experience for user, that he/she will come to your page for looking for something, and we are showing them something else(we may be not relevent to what he/she is looking for)?
    – yajiv
    May 24, 2018 at 6:12
  • 4
    Redirecting non-existent pages to the homepage is bad for users and bots, and Google have said in the past they might see this as a soft 404.
    – Tim Fountain
    May 24, 2018 at 9:32
  • 2
    The user experience for a 410 or 404 are completely down to what appears on the page, which is within your control.
    – Tim Fountain
    May 24, 2018 at 12:09

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.