Questions tagged [crawl-errors]

Errors that a web crawler encounters while trying to crawl a site. In the case of Google's web crawler, these errors can be viewed by the webmaster in Google Search Console.

17 questions with no upvoted or accepted answers
Filter by
Sorted by
Tagged with
3 votes
0 answers
72 views

About sudden drop in crawl request of our site

Looking for some help from Google seo expert to help our company site to identify this urgent issue. Here is the thing: the crawl request of our site experience a sudden drop on 9th, Oct. And it hasn'...
3 votes
0 answers
232 views

Disallow Image Loading to Reduce Number of "Other Error" in Crawled Page Resources?

We have a hotel search website. Many landing pages of our site load resources asynchrously (e.g. Ajax), but very often (about 25% of the time) when we click "View Crawled Page" in Google Search ...
  • 161
2 votes
0 answers
132 views

Blocked due to other 4xx issue in Google Search Console

I have problems with error "Blocked due to other 4xx issue" in Google Search Console. I dont know how to fix that and what is problem. I just have on website "call button" for ...
2 votes
0 answers
333 views

How to inspect and fix Bing's "Crawl errors"

I signed up for Bing's Webmaster Tools and submitted my sitemap. The "search performance" blade shows this report with 12 crawl errors: Nowhere on the site could I find information about ...
2 votes
0 answers
36 views

When setting up a campaign, Facebook is crawling my website using an Iphone user agent

Whenever I'm I'm setting up a campaing on Facebook, I get two visits from Facebook's auto crawlers. The first one I can identify as a robot and serve the SSR (HTML) version to it. The user agent is: ...
2 votes
0 answers
32 views

URL crawlable in old Google search console, but blocked in new

Google stopped crawling images for our site, when we moved images to cdn subdomain. Old search console states image is crawlable, but new one states it is blocked by robots.txt (for same link). our ...
  • 255
2 votes
0 answers
43 views

Do crawl errors in Google Search Console have an expiration time

BACKGROUND In November, a former student contacted me that he had about 258 search errors in his Google Search Console. There were 352 pages on his WordPress site, so that is significant. As can be ...
1 vote
0 answers
42 views

Google won't index my newly created home page after I had used a removal request for the default WordPress site on my domain

I have a site built with WordPress. Initially, because the default content of the template was indexed in Google, I decided to remove those URLs through the removals section in the Google console ...
  • 11
1 vote
0 answers
29 views

Google say URL is blocked by robots.txt but it isn't

My Google search console says that my submitted URL is blocked by robots.txt, but I checked header, tags and robots.txt it isn't. My URL was on #1 for my target keyword and daily get 1000 clicks ...
1 vote
0 answers
1k views

JavaScript console errors in Google Search Console - is it still mobile-friendly?

I am seeing mobile usability issues in my Google Search Console report. When I tested the pages they were 'Mobile Friendly' but there were some page loading issues: page resources couldn’t be ...
  • 171
1 vote
0 answers
19 views

Google Search Console reports error pages when URL paths from one site are requested from another

I have two e-commerce websites. Both are built on Magento. In Google Webmasters Tools for one website it is showing an error with part of the URL from one website and the domain of the other. If my ...
1 vote
0 answers
118 views

Bing is not crawling and indexing my mobile subdomain

BACKGROUND I have a mobile site located at a sub-domain of m.hydro-cleansing.com. The site is working fine and can be accessed. I am using a search API provided by Bing that should perform a site ...
1 vote
0 answers
97 views

I still get "Googlebot can't access your site" messages after removing a site from Google Webmaster Tools

I had a client who moved their site a while back to another developer. When they moved it they took down the site and put up a coming soon page ( which is currently what's there). Googlebot warned me ...
0 votes
0 answers
38 views

In my Crawl Stats, 23% of requests are "Unauthorized" - is that bad?

Am I correct in thinking that having 23% of all crawl requests being blocked by "Unauthorized" 401/407 is NOT good? 68% of my crawl requests return 200 (OK) which is good of course. Is this ...
  • 405
0 votes
0 answers
56 views

Googlebot crawl anomaly due to blocked resources by Zopim

I am using Zopim live chat to enable chat support in my website, but Googlebot is showing error due to some unreachable resources from zopim service. I have attached a screen capture of the error. ...
0 votes
0 answers
300 views

Googlebot is unable to access my site's robots.txt

I am having this issue: "Google couldn't crawl your site because we were unable to access your site's robots.txt file." I have already tried all the suggestions provided in the related problems but ...
's user avatar
0 votes
0 answers
156 views

Indexed URL is very low comparing submitted URL in google

I have 84 URL in my website. Till 2 weeks back all the URL is indexed in Google. I have verified it in Google webmaster tools. But it was slightly decreased day by day. Now Google webmaster tools ...
  • 131