Questions tagged [crawl-errors]

Errors that a web crawler encounters while trying to crawl a site. In the case of Google's web crawler, these errors can be viewed by the webmaster in Google Search Console.

Filter by
Sorted by
Tagged with
2 votes
0 answers
132 views

Blocked due to other 4xx issue in Google Search Console

I have problems with error "Blocked due to other 4xx issue" in Google Search Console. I dont know how to fix that and what is problem. I just have on website "call button" for ...
2 votes
0 answers
333 views

How to inspect and fix Bing's "Crawl errors"

I signed up for Bing's Webmaster Tools and submitted my sitemap. The "search performance" blade shows this report with 12 crawl errors: Nowhere on the site could I find information about ...
3 votes
0 answers
72 views

About sudden drop in crawl request of our site

Looking for some help from Google seo expert to help our company site to identify this urgent issue. Here is the thing: the crawl request of our site experience a sudden drop on 9th, Oct. And it hasn'...
1 vote
0 answers
70 views

How to fix 5xx errors for wp-content URLs in Google Search Console? [closed]

For example, the following wp-content URL returns an HTTP 500 status code, and Google shows a 5xx error for it in Search Console: https://businezvalue.com/wp-content/plugins/jetpack/modules/custom-css/...
1 vote
0 answers
26 views

Mobile usability issues reported by Google at Search Console [duplicate]

Google search console reported that some pages have a Mobile Usability issues: Clickable elements too close together Content wider than screen Text too small to read Some details: this is not about ...
0 votes
0 answers
38 views

In my Crawl Stats, 23% of requests are "Unauthorized" - is that bad?

Am I correct in thinking that having 23% of all crawl requests being blocked by "Unauthorized" 401/407 is NOT good? 68% of my crawl requests return 200 (OK) which is good of course. Is this ...
  • 405
0 votes
1 answer
61 views

should I set images to no index to avoid crawl budget limit

Most of my posts in search console are showing URL is on Google, but has issues I checked Crawl Page more info section and found out the blog post with many images show other error and the image ...
1 vote
0 answers
42 views

Google won't index my newly created home page after I had used a removal request for the default WordPress site on my domain

I have a site built with WordPress. Initially, because the default content of the template was indexed in Google, I decided to remove those URLs through the removals section in the Google console ...
  • 11
1 vote
0 answers
29 views

Google say URL is blocked by robots.txt but it isn't

My Google search console says that my submitted URL is blocked by robots.txt, but I checked header, tags and robots.txt it isn't. My URL was on #1 for my target keyword and daily get 1000 clicks ...
2 votes
1 answer
267 views

Remove large amounts of pages from Bing Index Explorer

I have a website that I took over in my business. We redid the whole website (which was previously a mix of ASP and HTML) and is now a WordPress site running on nginx. I submitted the sitemap to ...
  • 121
3 votes
0 answers
232 views

Disallow Image Loading to Reduce Number of "Other Error" in Crawled Page Resources?

We have a hotel search website. Many landing pages of our site load resources asynchrously (e.g. Ajax), but very often (about 25% of the time) when we click "View Crawled Page" in Google Search ...
  • 161
0 votes
1 answer
508 views

Google Search Console: issues detected "Top Errors" Server Error (5xx). BUG?

Just got this email from Google Search Console: This were the issues: But when I inspected those live URL's (using Google Search Console tool) they came back as "available to Google" and one of them ...
2 votes
1 answer
105 views

Google Search Console Warning: " Indexed, though blocked by robots.txt " (BUG)

Two days ago I've got an email with a warning from Google Search Console, telling me that 3 URLs from my site where indexed, but were blocked by robots.txt. Since I first deployed my website, I haven'...
2 votes
0 answers
36 views

When setting up a campaign, Facebook is crawling my website using an Iphone user agent

Whenever I'm I'm setting up a campaing on Facebook, I get two visits from Facebook's auto crawlers. The first one I can identify as a robot and serve the SSR (HTML) version to it. The user agent is: ...
2 votes
1 answer
103 views

Why is Google making up pages to crawl on my site?

For some reason Google is listing a series of pages that do not exist on my website such as: https://www.my_domain.com/index.php/about_us.php It lists them as 'Duplicate, Google chose different ...
2 votes
1 answer
242 views

Page cannot be reached in Google Rich snippet test

When i test my webpage in google rich test, it say page cannot be reached. My website is on shopify so i guess there should be no issue from server side. I need your help please... I cannot figure out ...
  • 21
2 votes
0 answers
32 views

URL crawlable in old Google search console, but blocked in new

Google stopped crawling images for our site, when we moved images to cdn subdomain. Old search console states image is crawlable, but new one states it is blocked by robots.txt (for same link). our ...
  • 255
1 vote
0 answers
696 views

How to allow only admin-ajax.php to be crawled? [closed]

My robots.txt looks as follows: User-agent: * Disallow: /wp-admin/ Allow: /wp-admin/admin-ajax.php Google Webmaster Tool recently raised a warning saying: Indexed, though blocked by robots.txt I ...
  • 121
2 votes
1 answer
962 views

Google Search Console still reporting 5xx errors even when my server has been running flawlessly for 7 days

For the past few days, I see that I have the following index coverage issue: Last month I was working on my production server and during this time the production server threw some errors (this ...
2 votes
1 answer
72 views

Getting usability issues in Google Search Console for the responses of API calls

Today I found an error that appeared in GSC recently. The URL of the affected page is not a page actually. That’s a example.com/api/v2/getcurrentuser API call that the JS is requesting to retrieve ...
0 votes
0 answers
56 views

Googlebot crawl anomaly due to blocked resources by Zopim

I am using Zopim live chat to enable chat support in my website, but Googlebot is showing error due to some unreachable resources from zopim service. I have attached a screen capture of the error. ...
1 vote
1 answer
131 views

Yoast purge plugin caused crawl errors, getting more even after removing it

I'm using the Yoast SEO plugin on my WordPress site, and when you upload media to WordPress media library, it makes a copy of the image or file and creates a file path (this is normal). However, all ...
1 vote
1 answer
172 views

Google can't access my page due to robots.txt but the testing tool says it's OK

My robots.txt is the typical that allows all bots everywhere User-agent: * Disallow: / When I test it with Google tool everything is OK. But in Google Search Console I get an error that my site can'...
0 votes
1 answer
242 views

Why are bots or servers crawling non- (NEVER) existent URLs and how can I stop this?

Through Google Analytics I started noticing hits in the dozens via a source page named as "free-seo-help.org" that redirects to http://seo2.agri-co-jsy.com/?site=http:// It does not go to my actual ...
1 vote
1 answer
184 views

404 on google cached version pages [duplicate]

When I am searching my website on Google cache version is showing 404 errors. I tested same for some other competitors also and the same problem is there. Reviewed on Google Search Console, all good ...
4 votes
1 answer
1k views

How can I "mark as fixed" in Google's new Search Console

In the "old" Google Search Console (and Google Webmaster Tools) there was an option to Mark as fixed to clear errors once fixed. This allowed us to keep the error list clean and only show errors that ...
  • 5,814
1 vote
2 answers
2k views

How to fix 302 errors in my Google Search Console?

I have so many 302 error in my Google Search Console. I changed HTTP to HTTPS recently and confusing part in Google Search Console is when I check there's /search.php on the end of every URL. It ...
0 votes
1 answer
316 views

Google Search Console Crawl 404 linked Error from strange Website [duplicate]

Image: The error is displayed because the URL has added the symbol ; at the end. /wp-content/uploads/2018/06/iuiuiuiuiuiui.pdf; (incorrect) If we remove the symbol ; the URL it works normally. /...
  • 31
1 vote
0 answers
1k views

JavaScript console errors in Google Search Console - is it still mobile-friendly?

I am seeing mobile usability issues in my Google Search Console report. When I tested the pages they were 'Mobile Friendly' but there were some page loading issues: page resources couldn’t be ...
  • 171
2 votes
1 answer
595 views

404 Crawl Error to almost all pdf files

The website is built in wordpress. I do not have than much experience, so I need your help. Once I entered at “Search Console” at “Crawl Errors” tab I saw that all “404 errors” are in “wp-content/...
  • 31
6 votes
3 answers
3k views

Google Search Console reports blocked resources from DoubleClick while rendering my pages, where do these calls come from?

I'm seeing errors when rendering pages: Googlebot couldn't get all resources for this page. Here's a list: url: https://static.doubleclick.net/instream/ad_status.js type: Script reason: Blocked ...
  • 71
1 vote
1 answer
46 views

After changing WordPress themes, search console reports 404s for existing URLs that can be rendered as well as many images

The site is now showing increasing numbers of 404s for the URL that are there - both in the URL and can be fetched and rendered. The problems started about two weeks ago with the Divi theme - it wasn'...
  • 71
1 vote
1 answer
145 views

Google Search Console is reporting an hAtom error for every page on my site

Whenever I publish a new article to my site, Google reports a new error by default in Search Console. I don't see what the issue is. How do I fix this?
  • 11
14 votes
3 answers
36k views

What is the solution for "Submitted URL not selected as canonical" in Google Search Console beta?

Google's new webmaster tools has a section 'Index coverage'>'Excluded'>'Submitted URL not selected as canonical'. I am getting 3 internal links in the section "Submitted URL not selected as ...
  • 141
3 votes
1 answer
273 views

How do I do an htaccess rewrite on an internal URL without Google crawl giving me a soft 404 error?

Here is my issue. I have a rewrite rule in my htaccess like this: RewriteRule ^(news|review|feature|editorial|podcast)/([0-9]+)/(.*)$ /article.php?id=$2 [L] Basically what it does is take in a URL ...
1 vote
1 answer
84 views

Outdated 404 errors that are linked to from the pages themselves

I have a lot of 404 errors in my Google search console and have been researching this a bit. I understand they are generally not going to impact anything, but I did read that if they are internally ...
  • 11
1 vote
0 answers
19 views

Google Search Console reports error pages when URL paths from one site are requested from another

I have two e-commerce websites. Both are built on Magento. In Google Webmasters Tools for one website it is showing an error with part of the URL from one website and the domain of the other. If my ...
6 votes
1 answer
2k views

Search Console: Server Connectivity Error - Connect Timeout

Over the last 4 days server connectivity errors have started to pop up in my Search Console. Specifically, it is the connection timeout error. I have been talking to my hosting provider and they ...
1 vote
1 answer
561 views

Google crawler 520 error

Google Crawler generates 520 errors daily. I will get about 50-100 of these errors in a 60.000 internal links website. The thing is though, that my links don't have any issues that I can dig into ...
  • 335
2 votes
1 answer
1k views

Why is Google crawling non-existant URLs?

I can see in live traffic of my wordpress website that goggle bot crawl non existing pages. www.example.gr/search/search-results/password-reset%252Fpassword-reset/password-reset%252Fpassword-reset%...
2 votes
1 answer
79 views

Old category pages with URL parameters

I have a problem with already indexed URLs (100k URLs). I have correct the pages with noindex and canonical tags over a month but old indexed pages aren’t removed from results. The problem is the URL ...
0 votes
1 answer
33 views

Google Search Console displays errors with ":fast_forward" added to URLs

I have two webpages and their URLs are like: https://example.com/videoCenter https:/example.com/register But today I found that indexing errors show that Google had crawled URLs like: https://...
2 votes
0 answers
966 views

Wrong domain in google results [closed]

Google search results show wrong domain (that is on the same web server), i.e. site:rose.spb.ru shows pages from site:xn--h1adaodgdoce7g.xn--p1ai which result in 404 when clicked. Any help is ...
  • 21
0 votes
1 answer
93 views

Can 301 redirects to homepage solve 404 errors in Google Search Console?

I have a site with a few hundred 404s. I am sure there are no relevant pages to redirect users to so I want to redirect users to homepage. Will redirecting to homepage be a good strategy compared to ...
's user avatar
2 votes
0 answers
165 views

Sudden Spike in mobile app crawl errors - Search console [closed]

We are seeing something weird in our mobile app crawl dashboard on the search console. Up until 3rd Feb, we had only 3 crawl errors but suddenly that number is now 107k and it went up between 3rd ...
3 votes
2 answers
466 views

Google Search Console: 404 errors on existing pages

There is a few years old small website with very few pages (~5), which were indexed and ranked by Google. A few days ago 4 of those pages disappeared from SERPs and Search Console gives me 404 Not ...
  • 173
3 votes
0 answers
80 views

Google Search Console: existing sitemap not found [duplicate]

Google search console (at Crawl -> Sitemaps) gives me General HTTP error: 404 not found errors after I upload the sitemap.xml to my website. When I click the link generated in the search console, it ...
  • 173
2 votes
0 answers
43 views

Do crawl errors in Google Search Console have an expiration time

BACKGROUND In November, a former student contacted me that he had about 258 search errors in his Google Search Console. There were 352 pages on his WordPress site, so that is significant. As can be ...
1 vote
1 answer
624 views

404 attack with non existent URLs

Today I checked my google search console and found few interesting 404. All those pages never existed on my site. Funny thing is that it didn't scan for some vulnerable php files. It looked like some ...
  • 11
1 vote
1 answer
42 views

Google search console is reporting site errors where the "linking page" has never linked to that ever

There are many annoying Site Errors when I'm checking my Search Console at "Crawl > Crawl Errors". Here is a screenshot: Those links are totally made up and I am never including them in my sitemap, ...