1

enter image description here

My website's 301 response crawl requests has risen 10 times (from 150-250 to 2000-4000).

  1. What is/are the cause/s?
  2. Has anyone experienced it?
  3. Does it heart SEO in long term?
  4. What is the solution for that?

For your information, we have recently moved our site images to a subdomain CDN and there may be a connection but when I checked the URL returning that response there were no signs of image URLs.

1 Answer 1

2

I will assume you made no significant changes to your website recently that would explain this increase.

What is/are the cause/s?

Most likely, Google is recrawling old URLs that it had previously not crawled for a while. Looking at the date, this is probably the preparation for either the December 2022 Helpful Content Update or the December 2022 Linkspam Update. Google is known to refresh the index before updates.

Other scenarios are:

  • You made a change to your XML sitemap and Google can now discover more 301 redirect URLs
  • You made changes to your robots.txt and Google can now crawl more 301 redirect URLs
  • You created new 301 redirects
  • You have a system that automatically creates 301 redirects
  • You introduced a systematic issue in your internal linking where you link to a version of your website that is always redirected (http:// instead of https://, no www, no trailing slash). This can be a link that is invisible to humans, like rel-canonical, href-lang, or other header elements). It can also be alt-tags that try to load images wrongly. It could also be CSS files or other resources. This could also be triggered by a CMS update.
  • Your server configuration around adding/removing a trailing slash to every URL changed - maybe without your knowledge if you are on shared hosting.

The logical next step would be to drill down into the Google Search Console reports and look at example URLs. Maybe you can see a pattern. Google Search Console can also give you some information about how they discovered a specific URL.

You could also look at your log file, filter for Google bot requests, and see which 301 redirects were crawled by Google. You might need to perform a reverse IP lookup or compare against a set of known Google IP addresses to remove people who pretend to be the Google bot to crawl your website.

Has anyone experienced it?

Yes. Many times before Google Updates. And also specifically on December 4 of this year.

And sometimes for some of the other reasons mentioned above.

Does it heart SEO in long term?

Google recrawling old URLs does not do any harm. Neither short-term nor long-term. If it is something else, that could, in theory, cause harm.

What is the solution for that?

If it is just recrawling before/during a Google Update, the solution is to wait.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.