Assuming we have a page, which has as Status Code 200 OK
and as Response Headers Status HTTP/1.1 410 Gone
, like on image:
What is the behavior of bots / Googlebot? To crawl/index or not?
Webmasters Stack Exchange is a question and answer site for pro webmasters. It only takes a minute to sign up.
Sign up to join this communityAssuming we have a page, which has as Status Code 200 OK
and as Response Headers Status HTTP/1.1 410 Gone
, like on image:
What is the behavior of bots / Googlebot? To crawl/index or not?
Bots should see the same as what you are seeing in the browser and interpret the response in the same way. ie. as a 200 OK
. That is, after all, what the HTTP Status-Line (the first line of the response header) is reporting - which is part of the HTTP standard.
The Status
HTTP response header is non-standard. (It's less common to see the "HTTP version" specified in this field as well?)