Stack Exchange Network

Stack Exchange network consists of 174 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [crawl-errors]

The tag has no usage guidance.

1
vote
1answer
26 views

Google can't access my page due to robots.txt but the testing tool says it's OK

My robots.txt is the typical that allows all bots everywhere User-agent: * Disallow: / When I test it with Google tool everything is OK. But in Google Search Console I get an error that my site can'...
0
votes
1answer
50 views

Why are bots or servers crawling non- (NEVER) existent URLs and how can I stop this?

Through Google Analytics I started noticing hits in the dozens via a source page named as "free-seo-help.org" that redirects to http://seo2.agri-co-jsy.com/?site=http:// It does not go to my actual ...
0
votes
0answers
8 views

Google Search Console - URL Errors [duplicate]

For one of my clients, I am getting many URL errors which doesn't exist in my database, for example- In URL errors (others) - I am getting sitemap.php & sitemap.jsp, which doesn't exists, would be ...
1
vote
1answer
45 views

404 on google cached version pages [duplicate]

When I am searching my website on Google cache version is showing 404 errors. I tested same for some other competitors also and the same problem is there. Reviewed on Google Search Console, all good ...
3
votes
0answers
110 views

How can I “mark as fixed” in Google's new Search Console

In the "old" Google Search Console (and Google Webmaster Tools) there was an option to Mark as fixed to clear errors once fixed. This allowed us to keep the error list clean and only show errors that ...
1
vote
2answers
212 views

How to fix 302 errors in my Google Search Console?

I have so many 302 error in my Google Search Console. I changed HTTP to HTTPS recently and confusing part in Google Search Console is when I check there's /search.php on the end of every URL. It ...
0
votes
1answer
206 views

Google Search Console Crawl 404 linked Error from strange Website [duplicate]

Image: The error is displayed because the URL has added the symbol ; at the end. /wp-content/uploads/2018/06/iuiuiuiuiuiui.pdf; (incorrect) If we remove the symbol ; the URL it works normally. /...
0
votes
0answers
61 views

Proper transfer of atom feeds from Shopify to Wordpress

I recently change my site from Shopify to Wordpress. I notice that some URLs got this ".atom" on the end. I keep thing what generates this URL... I recently found the Shopify generates this page: ...
1
vote
0answers
307 views

JavaScript console errors in Google Search Console - is it still mobile-friendly?

I am seeing mobile usability issues in my Google Search Console report. When I tested the pages they were 'Mobile Friendly' but there were some page loading issues: page resources couldn’t be ...
2
votes
1answer
203 views

404 Crawl Error to almost all pdf files

The website is built in wordpress. I do not have than much experience, so I need your help. Once I entered at “Search Console” at “Crawl Errors” tab I saw that all “404 errors” are in “wp-content/...
0
votes
0answers
74 views

Search Console Shows Temporarily Unavailable for API Endpoints but Not When Called Directly

On our website, we have product pages, and on each product page, there is a service call that is made to fetch the price of that product based on the user's selected store. In Search Console, when ...
1
vote
1answer
384 views

Google Search Console reports blocked resources from DoubleClick while rendering my pages, where do these calls come from?

I'm seeing errors when rendering pages: Googlebot couldn't get all resources for this page. Here's a list: url: https://static.doubleclick.net/instream/ad_status.js type: Script reason: Blocked ...
1
vote
1answer
41 views

After changing WordPress themes, search console reports 404s for existing URLs that can be rendered as well as many images

The site is now showing increasing numbers of 404s for the URL that are there - both in the URL and can be fetched and rendered. The problems started about two weeks ago with the Divi theme - it wasn'...
0
votes
0answers
16 views

Search Console is reporting non-existent links related to a deleted WordPress plugin [duplicate]

I have problem with LayerSlider plugin. I see 20-30 pages in Crawl Errors and all from LayerSLider. I once had that plugin installed on my website. Here is an example: /wp-content/plugins/LayerSlider/...
0
votes
0answers
27 views

Google has crawled over 1000 404 product pages on my Magento 1 [duplicate]

For the last couple of months Google Search Console has reported an ever increasing number of 404 crawl errors on my Magento 1 products. It lists that it's linked from pages but I can't find any ...
1
vote
1answer
123 views

Google Search Console is reporting an hAtom error for every page on my site

Whenever I publish a new article to my site, Google reports a new error by default in Search Console. I don't see what the issue is. How do I fix this?
5
votes
2answers
8k views

What is the solution for “Submitted URL not selected as canonical” in Google Search Console beta?

Google's new webmaster tools has a section 'Index coverage'>'Excluded'>'Submitted URL not selected as canonical'. I am getting 3 internal links in the section "Submitted URL not selected as ...
3
votes
1answer
190 views

How do I do an htaccess rewrite on an internal URL without Google crawl giving me a soft 404 error?

Here is my issue. I have a rewrite rule in my htaccess like this: RewriteRule ^(news|review|feature|editorial|podcast)/([0-9]+)/(.*)$ /article.php?id=$2 [L] Basically what it does is take in a URL ...
1
vote
1answer
66 views

Outdated 404 errors that are linked to from the pages themselves

I have a lot of 404 errors in my Google search console and have been researching this a bit. I understand they are generally not going to impact anything, but I did read that if they are internally ...
0
votes
0answers
127 views

Webmaster tools 'URL errors' always shows errors

The issue I am facing is my webmaster tools always show an error message.The error URL's are not related to my website.The site url mentioned in the website tools are like siteurl/wyx or siteurl/docs....
0
votes
0answers
226 views

Website hacked : sitemap and URL totally weird

Okay so I have no idea how and where to start with this problem but it's been too long and I don't know what to do anymore. My website has been hacked a while ago, I've cleaned everything since then, ...
1
vote
0answers
18 views

Google Search Console reports error pages when URL paths from one site are requested from another

I have two e-commerce websites. Both are built on Magento. In Google Webmasters Tools for one website it is showing an error with part of the URL from one website and the domain of the other. If my ...
4
votes
0answers
1k views

Search Console: Server Connectivity Error - Connect Timeout

Over the last 4 days server connectivity errors have started to pop up in my Search Console. Specifically, it is the connection timeout error. I have been talking to my hosting provider and they ...
1
vote
1answer
289 views

Google crawler 520 error

Google Crawler generates 520 errors daily. I will get about 50-100 of these errors in a 60.000 internal links website. The thing is though, that my links don't have any issues that I can dig into ...
0
votes
0answers
20 views

Tons of gibberish crawl errors [duplicate]

My Wordpress site got hacked on Hostgator. I reinstalled everything from scratch but still get hundreds of very weird crawl errors that look like this: search/%E5%A4%A7%E5%A5%96%E8%80%81%E8%99%8E%...
2
votes
1answer
389 views

Why is Google crawling non-existant URLs?

I can see in live traffic of my wordpress website that goggle bot crawl non existing pages. www.example.gr/search/search-results/password-reset%252Fpassword-reset/password-reset%252Fpassword-reset%...
2
votes
1answer
68 views

Old category pages with URL parameters

I have a problem with already indexed URLs (100k URLs). I have correct the pages with noindex and canonical tags over a month but old indexed pages aren’t removed from results. The problem is the URL ...
0
votes
1answer
31 views

Google Search Console displays errors with “:fast_forward” added to URLs

I have two webpages and their URLs are like: https://example.com/videoCenter https:/example.com/register But today I found that indexing errors show that Google had crawled URLs like: https://...
1
vote
0answers
638 views

Wrong domain in google results [closed]

Google search results show wrong domain (that is on the same web server), i.e. site:rose.spb.ru shows pages from site:xn--h1adaodgdoce7g.xn--p1ai which result in 404 when clicked. Any help is ...
0
votes
1answer
78 views

Can 301 redirects to homepage solve 404 errors in Google Search Console?

I have a site with a few hundred 404s. I am sure there are no relevant pages to redirect users to so I want to redirect users to homepage. Will redirecting to homepage be a good strategy compared to ...
2
votes
0answers
139 views

Sudden Spike in mobile app crawl errors - Search console [closed]

We are seeing something weird in our mobile app crawl dashboard on the search console. Up until 3rd Feb, we had only 3 crawl errors but suddenly that number is now 107k and it went up between 3rd ...
3
votes
1answer
336 views

Google Search Console: 404 errors on existing pages

There is a few years old small website with very few pages (~5), which were indexed and ranked by Google. A few days ago 4 of those pages disappeared from SERPs and Search Console gives me 404 Not ...
3
votes
0answers
60 views

Google Search Console: existing sitemap not found [duplicate]

Google search console (at Crawl -> Sitemaps) gives me General HTTP error: 404 not found errors after I upload the sitemap.xml to my website. When I click the link generated in the search console, it ...
2
votes
0answers
38 views

Do crawl errors in Google Search Console have an expiration time

BACKGROUND In November, a former student contacted me that he had about 258 search errors in his Google Search Console. There were 352 pages on his WordPress site, so that is significant. As can be ...
0
votes
1answer
334 views

404 attack with non existent URLs

Today I checked my google search console and found few interesting 404. All those pages never existed on my site. Funny thing is that it didn't scan for some vulnerable php files. It looked like some ...
1
vote
1answer
36 views

Google search console is reporting site errors where the “linking page” has never linked to that ever

There are many annoying Site Errors when I'm checking my Search Console at "Crawl > Crawl Errors". Here is a screenshot: Those links are totally made up and I am never including them in my sitemap, ...
3
votes
2answers
372 views

Bing webmaster indicating crawl errors on http to https 301 redirect

I moved my site form http to https, now sitemap contains https urls, but Bing webmaster indicates increased "crawl errors" and my indexed pages have decreased to 0. The list of errors says http pages ...
0
votes
1answer
51 views

Google notified me that I have a sudden spike in 404's, but they are pages that don't exist

I'm pretty puzzled by this, so any direction would be helpful. The only thing hosted on this server is basically two word press instances with my personal site and another. Everything's been great, ...
0
votes
2answers
203 views

Google crawl errors after deleting old site

I deleted my old site entirely and made a new site 8 months ago. But still in the crawl errors section of Google Search Console, a lot of 404 and not found errors appear. All the URLs are for the ...
1
vote
1answer
70 views

Will Google remove links restricted by the robots.txt from the soft 404 list?

I have updated a shop system to a newer version. I now get huge number of soft 404 errors in the Webmaster Tools because of links that have been indexed in the old version of that shop. I could lock ...
0
votes
2answers
69 views

Error HTTP 404 can eliminate indexing of a page?

In recent weeks I'm recording a decrease of indexing my site: In 10/09/2016: Submitted => 921 Indexed => 654 Yesterday 23/09/2016: Submitted => 1129 Indexed => 598 Two months ago I created a ...
5
votes
1answer
145 views

Can Googlebot discover URLs that are not in an anchor?

In my Search Console I had a sudden increase in 404's. They were linked from a webpage that listed files from the site in a table but none of these URLs were nested inside an <a> tag. This was ...
4
votes
1answer
1k views

Google reports a sudden increase in 404 errors from crawling pages

My server went down for a couple days.(2-3 august) 15 days later, Google Webmaster Tools notified me of increase in 404 errors. The problem is that Google reports 70,000 error pages while it has ...
1
vote
1answer
70 views

Google Websmaster Tools crawl errors reporting a link that does not exist

Google Webmaster Tools seems to be giving me an erroneous report. The crawl Errors (Smart Phone tab) still shows a link to http://mypubguide.com/good-pubs/Blacko from http://mypubguide.com/good-pubs/...
3
votes
3answers
539 views

What to do with unwanted URLs that link to my website?

A few days ago, my site was affected by malware. There were lot of URLs that linked to my site and the malware used to redirect to some place. I have gotten rid of all the malware, but still there are ...
2
votes
2answers
67 views

When would Google crawler give up crawling a URL with 5xx error

How many times will Google crawler attempt to recrawl a URL that has a 5xx error occurring intermittently? When would it give up and drop the URL from the Index?
2
votes
1answer
91 views

Google sitelinks not getting generated

Even though our website is indexed by google, sitelinks are not getting generated. It's been almost 2 years now and still not sitelinks. Google webmaster tools has a lot of 404 errors. Are these 404 ...
3
votes
1answer
241 views

How does Google rank crawl errors in Search Console

In Search Console (formerly Webmaster Tools) I am viewing the report at crawl > crawl errors. I see that Google has set a 'priority' for these 404s. I'm curious how Google determines what is a ...
0
votes
0answers
251 views

Googlebot is unable to access my site's robots.txt

I am having this issue: "Google couldn't crawl your site because we were unable to access your site's robots.txt file." I have already tried all the suggestions provided in the related problems but ...
1
vote
1answer
42 views

I get 404 error in google webmaster from unknown php page

In my wallpaper website that i code myself in php.There i get 404 error in google crawling.These errors are from unknow page like details.php?image_id=121&sessionid=...