The tag has no usage guidance.

learn more… | top users | synonyms

9
votes
6answers
15k views

Massive 404 attack with non existent URLs. How to prevent this?

The problem is a whole load of 404 errors, as reported by Google Webmaster Tools, with pages and queries that have never been there. One of them is viewtopic.php, and I've also noticed a scary number ...
6
votes
1answer
2k views

How does Google Rank “Priority” for “Not Found” errors?

Google lists Not Found Errors in Webmaster Tools as Ranked by Priority, but what is Priority based on?
4
votes
1answer
75 views

How to tell Googlebot to crawl just URLs ending in .html

I have a big problem in Google Webmaster Tools. The number of 404 pages in the error report increases so fast that now I have over 1000. When I check for errors, I see that for every page Googlebot ...
4
votes
1answer
366 views

Weird URLs being access by Googlebot

Lately I've been seing all sorts of strange URLs show up as errors in my Webmaster tools account, but they're URLs that don't actually exist on my site, nor are linked from the pages that Google ...
3
votes
2answers
224 views

What to do with unwanted URLs that link to my website?

A few days ago, my site was affected by malware. There were lot of URLs that linked to my site and the malware used to redirect to some place. I have gotten rid of all the malware, but still there are ...
3
votes
1answer
1k views

URL/Crawl Errors continue to increase after pharma hack

I'm working with a site that was hacked a couple of months ago and was sending out a ton of spam. The problem seemed to have started with a dormant WordPress blog that was part of the site (so the ...
3
votes
1answer
457 views

Google Webmasters Tools strange 404 errors referred from same site

Starting about a month ago, I noticed a sudden increase in 404 errors in Webmasters Tools for one of my sites (over 1400 errors so far). All the errors are being referred from my own site to non ...
3
votes
2answers
83 views

How to set crawl rate in Google Webmaster Tools for seven days or a month?

My web host is blocking IPs which make many requests. This includes Googlebot. I found this by checking my webmaster tools account. There it said "network unreachable" for all the URLs which are ...
3
votes
1answer
29 views

How does Google rank crawl errors in Search Console

In Search Console (formerly Webmaster Tools) I am viewing the report at crawl > crawl errors. I see that Google has set a 'priority' for these 404s. I'm curious how Google determines what is a ...
3
votes
1answer
315 views

Resetting sitemap warnings in Webmaster Tools

Back in the day when I was updating my website and had errors, Google Webmaster Tools took note of it and gave my sitemap files warnings. Now I decided to start over new again by uploading brand new ...
3
votes
3answers
105 views

Crawl errors “src” in the last in URL

I was looking in the Google Webmaster Tools and realized the following crawl error: Pages that link to http://mysite.com/articles/my-stub/src/ URL: http://mysite.com/articles/my-stub I wonder ...
3
votes
2answers
73 views

Increase in 5XX errors and 404 errors on GWT, are we under attack?

A lot of URLs are suddenly giving us 404s and 500s. On closer inspection, these URLs do not exist on our website and are being linked from random sources. As more URLs were inspected on GWT, a ...
3
votes
1answer
57 views

Fetch as Google says my page is “Temporary Unreachable”

When looking into it in WebmasterTools, I saw that there is a "Fetch as Google - Temporarily Unreachable Issue". It is for an image, and font. How do I resolve this issue?
3
votes
1answer
418 views

SEO impact of server errors

Our website is up and running fine but a WordPress plugin is broken (we can't fix it very quickly) so a few hundred pages show server errors. The pages would load header and page of the page body but ...
3
votes
2answers
132 views

Yandex frequently replaces page names with ampersands

The Yandex spider is a frequent visitor to one of the sites I manage. On occasion it replaces the page name with two ampersands and a space. So if the page is: /mypage.aspx?param=value then it ...
2
votes
3answers
1k views

Weird 404 crawl errors in Google webmaster tools

If anyone can help, I'd be very grateful! We're getting strange domains coming up in Google webmaster tools - they seem to be breeding. When I look where the links are coming from, it's either: ...
2
votes
2answers
3k views

Why are there Google crawl errors for unknown urls?

I have been getting crawl errors in Google's WebMaster tools for urls like the following: http://madisonps.com/Media/eurl.axd/812e5d58643d804993276b6417a3c5ef These urls are not part of our site and ...
2
votes
2answers
36 views

When would Google crawler give up crawling a URL with 5xx error

How many times will Google crawler attempt to recrawl a URL that has a 5xx error occurring intermittently? When would it give up and drop the URL from the Index?
2
votes
2answers
46 views

Google crawler accessing non existing pages

I am looking Search Console and found Crawl errors for following URLs on my site: example.com/mobile example.com/m I never mentioned these pages anywhere but why Google is accessing it and what ...
2
votes
1answer
214 views

How do I fix the crawl errors from Google Search Console where all my URLs end with htmlprint instead of html?

I moved from Blogger to Wordpress. I set my prefered domain to non-www in webmaster tools. I submitted a sitemap. My website is OK when I browse it. Links from other sites to my pages are well ...
2
votes
2answers
958 views

Google Webmaster Tools reporting non-existent links

GWT is reporting 404 crawl errors on pages that have never been part of the website. For example, it's reporting crawl errors on several non-existent pages such as mysite.net/blog/how-can-i-improve-...
2
votes
1answer
146 views

Google Webmaster Crawl Stats Missing

I added my website in Google Webmaster Tools two months ago and in "Crawl Stats" section I am seeing the following data. Pages crawled per day - Missing graph data for some specific date And the same ...
2
votes
3answers
292 views

Google Webmaster Tools keeps reporting not finding pages from the old website

About 3 months ago we've completely replaced the content of our domain, practically building up a new website. The sitemap for the new website has been submited to Google Webmaster Tools, the website ...
2
votes
1answer
525 views

Google Webmaster Tools is reporting crawl errors for links that don't exist

In Google webmaster tools > diagnostics > crawl errors I'm seeing a bunch of errors that link to sub-pages like /tour/overview/Scripts/ForceDownload.asp The /tour/overview does exist, but this ...
2
votes
2answers
176 views

Will setting a custom crawl rate in Google Webmaster Tools affects index or have other negative consequences?

My site was running well for the past three years. Recently Googlebot's request my server got increased. and at one time I got network unreachable error. When I contacted my hosting company they asked ...
2
votes
2answers
365 views

Googlebot requesting invalid URL

I have a web app which emails me exceptions automatically. This morning there was an error relating to a url: /Catalog/LiveCatalog?id=ylwpfqzts id is invalid (should be a guid) and caused an error ...
2
votes
1answer
128 views

The previous owner of my domain hosted pirated content, what is the best course of action?

Judging by my error logs, the Google craler errors, and the massive amount of DCMA takedown notices I get it has become apparent that the previous owner of my domain name (ianspence.com) used to host ...
2
votes
2answers
1k views

Google webmaster Tool showing : 170 not found 404 errors!

I have a website which is 2 years old and it is simple PHP pages. I had set up Google Analytics and Google Webmaster Tools for it. Just last week I removed that PHP website and installed a new ...
2
votes
2answers
564 views

Google Webmaster Tools “increase in not found errors” is wrong

I got a ton of not found errors in Google Webmaster Tools, but the weird thing is, all of the pages ARE found. There is absolutely no error in the URLs they list that supposedly don't exist. Why is ...
2
votes
3answers
628 views

GWT: Generate more complete crawl error report

I'm a developer in charge of managing Webmasters and related issues (including correcting crawl errors) for dozens (hundreds, maybe?) of active sites and as part of my duties I create a report of ...
2
votes
3answers
7k views

How to resolve “Google can't find your site's robots.txt” error?

I've recently found that "Google can't find your site's robots.txt" in crawl errors. When I tried Fetching as Google, I got result "SUCCESS", then I tried looking at crawl errors and it still shows "...
2
votes
3answers
133 views

Can server errors cause major traffic drop from Google?

My web server was down for about 30 min. The next day (30~ hours from the event) I noticed major drop in traffic from Google. The normal traffic to my site is about 2500 unique/day and after the event ...
2
votes
1answer
29 views

Google sitelinks not getting generated

Even though our website is indexed by google, sitelinks are not getting generated. It's been almost 2 years now and still not sitelinks. Google webmaster tools has a lot of 404 errors. Are these 404 ...
2
votes
1answer
725 views

How to Keep “Good” Bots That Ignore Robots & Nofollow From Getting Into Honeypots?

We have a self generating honeypot that is designed to fill crawlers with junk data (among other things). Its protected with the right "treaties" and headers so 99% of good bots stay away. Today ...
2
votes
4answers
427 views

In Google webmaster tools, can a “soft 404” be triggered by the text on the page?

I just ran across an error in Google Webmaster Tools that I have never seen before. I manage the website for my local community band (I play trombone). One of the pages on the site is a list of ...
2
votes
2answers
920 views

How does Google treat unlinked URLs?

In Google Webmaster Tools I've recently noticed several concatenated URLs appearing in the Crawl Errors. On investigation it seems Google is picking up these links from a page that shows a user's ...
2
votes
1answer
444 views

Website not fetched by Bingbot: Unexpected Error

My website has been online for a couple years. It's on the first result for Google but it doesn't seem to be indexed by Bing. I tried to fetch it with bing in their webmaster tools and it always ...
2
votes
2answers
1k views

Large increase in 404 hits

We're running a Magento shop and we experienced a significant raise in 404 hits lately, almost 40,000 in one day. Most of the pages are old ones that once existed but don't anymore, and there are some ...
1
vote
3answers
1k views

“High response time” warnings in Google Webmaster Tools

In my Google Webmaster Tools account, there are warnings about my site. Some URLs listed in this sitemap have a high response time. This may indicate a problem with your server or with the content ...
1
vote
3answers
4k views

How to fix robots.txt file error in my GWT?

In my blog’s Webmaster Tools, there is a notification in Crawl Errors section, that is Google couldn't crawl your site because we were unable to access the robots.txt file. My blog’s robots.txt file ...
1
vote
1answer
176 views

“Access denied” error in Google Webmaster tools

I have an access denied problem in my webmaster tools. It says 404 errors. How can I solve this? I have already fixed the errors, but they are not removed from Google Webmaster Tools.
1
vote
1answer
214 views

Google Webmaster Tools reported crawl errors due to DNS, server connectivity, and robots.txt unavailable

I am experiencing the following site errors on Google Webmaster Tools. DNS: couldn’t communicate with the DNS server. Server connectivity: request timed out or site is blocking Google. Robots.txt ...
1
vote
2answers
451 views

Crawler errors for unused URLs containing “mozekcdn-a.akamaihd.net”

In the AdSense dashboard, Google is reporting crawl errors for unexpected URLs. It shows that Google crawler was unable to access some of my 87 pages. When I checked those pages I found that I haven't ...
1
vote
2answers
81 views

What to do about crawler errors after removing non-essential parts of my website?

So, recently I was re-building my website, putting on another CMS, optimizing and so on. Now I visited Google Webmaster tools and it tells I have a lot of Web crawler 404 not found errors. I am ...
1
vote
1answer
44 views

How to avoid google from indexing like example.com/post/post/post/1-foobar.html?

I have a weird anomaly with one project. Where Google is indexing sites like: example.com/post/post/post/1-foobar.html While it should do it like: example.com/post/1-foobar.html I just ...
1
vote
1answer
16 views

Google Websmaster Tools crawl errors reporting a link that does not exist

Google Webmaster Tools seems to be giving me an erroneous report. The crawl Errors (Smart Phone tab) still shows a link to http://mypubguide.com/good-pubs/Blacko from http://mypubguide.com/good-pubs/...
1
vote
1answer
119 views

Can't find 404's that are listed

I've been asked to look at the "Not Found" tab in Google Webmaster Tools, to try and eliminate as many of the thousands of 404's not listed. There are several reasons for these, but this post will ...
1
vote
1answer
83 views

Crawl errors appearing in Google Webmaster Tools with suspicious URLs cached

I am receiving Crawl Errors in my Google Webmaster Tools reports with some unique URLs that I am not understanding. For Example: http://www.example.com/portfolio/?cat_url=custom-cms This page ...
1
vote
1answer
601 views

Google WebMaster Tools Reports a Significant Increase in 500 Response Code Errors

Google's Webmaster tools shows lots of Crawl Errors. They are reported as 500 response code. How can I go about diagnosing this issue? The website is eCommerce business using the Magento platform....
1
vote
1answer
56 views

Trendmicro clones authenticated requests - showing up in my error logs

I'm having various errors, from unauthenticated requests for using some AJAX requests (which need authentication). While I have clients from Belgium, the IP addresses match a IP in the form of 150.70....