The tag has no wiki summary.

learn more… | top users | synonyms

2
votes
1answer
56 views

Google Webmaster Tools reporting non-existent links

GWT is reporting 404 crawl errors on pages that have never been part of the website. For example, it's reporting crawl errors on several non-existent pages such as ...
2
votes
1answer
36 views

Resetting sitemap warning count in webmaster tools

Back in the day when I was updating my website and had errors, Google Webmaster Tools took note of it and gave my sitemap files warnings. Now I decided to start over new again by uploading brand new ...
1
vote
1answer
57 views

Can't find 404's that are listed

I've been asked to look at the "Not Found" tab in Google Webmaster Tools, to try and eliminate as many of the thousands of 404's not listed. There are several reasons for these, but this post will ...
1
vote
1answer
79 views

Google Webmaster Tools reported crawl errors due to DNS, server connectivity, and robots.txt unavailable

I am experiencing the following site errors on Google Webmaster Tools. DNS: couldn’t communicate with the DNS server. Server connectivity: request timed out or site is blocking Google. Robots.txt ...
1
vote
2answers
424 views

Crawler errors for unused URLs containing “mozekcdn-a.akamaihd.net”

In the AdSense dashboard, Google is reporting crawl errors for unexpected URLs. It shows that Google crawler was unable to access some of my 87 pages. When I checked those pages I found that I haven't ...
1
vote
0answers
39 views

Bing is not crawling and indexing my mobile subdomain

BACKGROUND I have a mobile site located at a sub-domain of m.hydro-cleansing.com. The site is working fine and can be accessed. I am using a search API provided by Bing that should perform a site ...
1
vote
1answer
99 views

How to fix Adsense error: “Our crawler was unable to access the pages” when those pages had been removed and are 404 not found?

I have a website and we recently changed content management systems. With the new system came new URLs/hierarchy. Google Webmaster Tools doesn't have any issues with the site and I get no crawler ...
2
votes
1answer
146 views

Website not fetched by Bingbot: Unexpected Error

My website has been online for a couple years. It's on the first result for Google but it doesn't seem to be indexed by Bing. I tried to fetch it with bing in their webmaster tools and it always ...
2
votes
1answer
231 views

How to Keep “Good” Bots That Ignore Robots & Nofollow From Getting Into Honeypots?

We have a self generating honeypot that is designed to fill crawlers with junk data (among other things). Its protected with the right "treaties" and headers so 99% of good bots stay away. Today ...
0
votes
1answer
22 views

Comment Out Link Shows in Webmaster

I had "Privacy Policy" link on my old website. When I launched my new website, I put the link in footer but disallowed that to robots.txt by adding this code: <?php /*?><div ...
1
vote
2answers
50 views

What to do about crawler errors after removing non-essential parts of my website?

So, recently I was re-building my website, putting on another CMS, optimizing and so on. Now I visited Google Webmaster tools and it tells I have a lot of Web crawler 404 not found errors. I am ...
0
votes
1answer
57 views

Reset Connection causing multiple errors after hack

We use webmaster tools to monitor our websites, and in the past couple of months some of our sites have been compromised and had various pages inserted. These pages created URLs that Google would ...
0
votes
2answers
136 views

Google Webmaster Tools crawl errors due to Google decoding links that were urlencoded by Codeigniter

In our pages we have internal links that are urlencoded because the URL may sometime contain special characters such as () or ! So in our source code we have a link: ...
1
vote
1answer
57 views

Crawl errors appearing in Google Webmaster Tools with suspicious URLs cached

I am receiving Crawl Errors in my Google Webmaster Tools reports with some unique URLs that I am not understanding. For Example: http://www.example.com/portfolio/?cat_url=custom-cms This page ...
1
vote
1answer
32 views

How to avoid google from indexing like example.com/post/post/post/1-foobar.html?

I have a weird anomaly with one project. Where Google is indexing sites like: example.com/post/post/post/1-foobar.html While it should do it like: example.com/post/1-foobar.html I just ...
0
votes
4answers
72 views

Crawl errors and duplicate URL

I am getting lots of crawl errors, Google is adding main domain to the URL: sitemapxml: ...
1
vote
1answer
275 views

Google WebMaster Tools Reports a Significant Increase in 500 Response Code Errors

Google's Webmaster tools shows lots of Crawl Errors. They are reported as 500 response code. How can I go about diagnosing this issue? The website is eCommerce business using the Magento ...
7
votes
5answers
5k views

Massive 404 attack with non existent URLs. How to prevent this?

The problem is a whole load of 404 errors, as reported by Google Webmaster Tools, with pages and queries that have never been there. One of them is viewtopic.php, and I've also noticed a scary number ...
1
vote
1answer
22 views

Trendmicro clones authenticated requests - showing up in my error logs

I'm having various errors, from unauthenticated requests for using some AJAX requests (which need authentication). While I have clients from Belgium, the IP addresses match a IP in the form of ...
2
votes
1answer
108 views

The previous owner of my domain hosted pirated content, what is the best course of action?

Judging by my error logs, the Google craler errors, and the massive amount of DCMA takedown notices I get it has become apparent that the previous owner of my domain name (ianspence.com) used to host ...
1
vote
3answers
287 views

“High response time” warnings in Google Webmaster Tools

In my Google Webmaster Tools account, there are warnings about my site. Some URLs listed in this sitemap have a high response time. This may indicate a problem with your server or with the content ...
1
vote
0answers
81 views

I still get “Googlebot can't access your site” messages after removing a site from Google Webmaster Tools

I had a client who moved their site a while back to another developer. When they moved it they took down the site and put up a coming soon page ( which is currently what's there). Googlebot warned me ...
1
vote
1answer
85 views

Has Microsoft introduced a bug in Bingbot such that it crawls non-existent images?

I've started receiving requests for paths such as /scale/jackets/00000086/80x80_P2_G1715325W.JPG several times a day from the Bingbot. This started a couple of days ago. Anybody know why this is?
1
vote
1answer
107 views

“Access denied” error in Google Webmaster tools

I have an access denied problem in my webmaster tools. It says 404 errors. How can I solve this? I have already fixed the errors, but they are not removed from Google Webmaster Tools.
3
votes
1answer
577 views

URL/Crawl Errors continue to increase after pharma hack

I'm working with a site that was hacked a couple of months ago and was sending out a ton of spam. The problem seemed to have started with a dormant WordPress blog that was part of the site (so the ...
1
vote
2answers
957 views

Why are there Google crawl errors for unknown urls?

I have been getting crawl errors in Google's WebMaster tools for urls like the following: http://madisonps.com/Media/eurl.axd/812e5d58643d804993276b6417a3c5ef These urls are not part of our site and ...
2
votes
4answers
286 views

In Google webmaster tools, can a “soft 404” be triggered by the text on the page?

I just ran across an error in Google Webmaster Tools that I have never seen before. I manage the website for my local community band (I play trombone). One of the pages on the site is a list of ...
-1
votes
1answer
123 views

Google crawling/indexing PHP files

All of a sudden I had about 819 errors pop up in my Webmaster Tools accounts, all of which are PHP files. Google has never indexed these before, because they aren't URLs. The main folder they are ...
1
vote
2answers
869 views

Google webmaster Tool showing : 170 not found 404 errors! [duplicate]

I have a website which is 2 years old and it is simple PHP pages. I had set up Google Analytics and Google Webmaster Tools for it. Just last week I removed that PHP website and installed a new ...
2
votes
2answers
401 views

Google Webmaster Tools “increase in not found errors” is wrong

I got a ton of not found errors in Google Webmaster Tools, but the weird thing is, all of the pages ARE found. There is absolutely no error in the URLs they list that supposedly don't exist. Why is ...
1
vote
2answers
113 views

How do I diagnose 408 crawl errors?

Somebody sent me a message about my website: "I'm getting a 408 error when I try to crawl it." How do I resolve the 408 errors that are causing problems for this crawler?
0
votes
2answers
173 views

Google Webmaster Tools crawl error with 'function.session-start' URLs

I've got following error URLs from the Google Webmaster Tools crawl errors. http://www.mydomain.com/mydomain/shirts/function.session-start I've checked the those pages that links from but I didn't ...
0
votes
1answer
99 views

How to fix unknown broken link error in WordPress?

I've been working on a client website and found that there are broken links. I have fixed most of the links. Most of the broken links are generated from comments. So I fixed them. But still when I ...
2
votes
1answer
121 views

Google Webmaster Crawl Stats Missing

I added my website in Google Webmaster Tools two months ago and in "Crawl Stats" section I am seeing the following data. Pages crawled per day - Missing graph data for some specific date And the same ...
1
vote
1answer
188 views

Incomprehensible new 403 errors from Google

I recently received repeated notices from Google saying my site has an increase in 403 authorization permission errors. Looking at the chart I see that starting on July 22 the errors are indeed ...
0
votes
3answers
3k views

How to fix robots.txt file error in my GWT?

In my blog’s Webmaster Tools, there is a notification in Crawl Errors section, that is Google couldn't crawl your site because we were unable to access the robots.txt file. My blog’s robots.txt file ...
0
votes
1answer
100 views

How can I create a sitemap for 2000 products powered by a single php page?

Usually I generate sitemaps using www.xml-sitemaps.com and the limit is 500 pages. My website has barely 30, but I have a lot of products in my database, so the page product.php becomes ...
1
vote
3answers
502 views

Too many “access denied” errors showing in Google Webmaster Tools

I get 18,000 "access denied" errors showing in Google Webmaster Tools every day! It's strange that it shows for URL's with www but not for those without it (i.e., just domain.com instead of ...
0
votes
1answer
72 views

What is the difference between not found and not followed in Google Webmaster Tools?

In Google Webmaster Tools under Crawl Errors I found 18 Not Found and 16 Not Followed errors. What does this mean, and what is the difference between them?
2
votes
3answers
249 views

Google Webmaster Tools keeps reporting not finding pages from the old website

About 3 months ago we've completely replaced the content of our domain, practically building up a new website. The sitemap for the new website has been submited to Google Webmaster Tools, the website ...
0
votes
1answer
567 views

404 errors How to deal with a large number of broken internal links for SEO

I have recently updated my website. In fact I have completely rewritten it using wordpress. It was originally constructed using ASP. In the process, I have recycled a a large amount of my original ...
0
votes
2answers
268 views

Redirect pages to fix crawl errors

Google is giving me a crawl error for pages that I have removed like www.mysite.com/mypage.html. I want to redirect this pages to the new page www.mysite.com/mysite/mypage. I tried to do that by ...
0
votes
1answer
128 views

Spiders crawling broken links

Tl:Dr Unknown bot crawling the same broken (HTTP 400) URL over and over again. Different User Agent and different country of origin. The Problem It seems at least once a week we're getting a big ...
2
votes
2answers
714 views

Large increase in 404 hits

We're running a Magento shop and we experienced a significant raise in 404 hits lately, almost 40,000 in one day. Most of the pages are old ones that once existed but don't anymore, and there are some ...
2
votes
3answers
508 views

GWT: Generate more complete crawl error report

I'm a developer in charge of managing Webmasters and related issues (including correcting crawl errors) for dozens (hundreds, maybe?) of active sites and as part of my duties I create a report of ...
1
vote
3answers
1k views

Weird 404 crawl errors in Google webmaster tools

If anyone can help, I'd be very grateful! We're getting strange domains coming up in Google webmaster tools - they seem to be breeding. When I look where the links are coming from, it's either: ...
1
vote
3answers
3k views

Google Webmaster Tools - Crawl errors with 500 status code

I will try to explain the situation: In December 2012 I deployed a website with a sitemap.xml, etc... there were circa 2000 URLs. After a while I changed something on my website, upgrading my pages ...
3
votes
1answer
331 views

SEO impact of server errors

Our website is up and running fine but a WordPress plugin is broken (we can't fix it very quickly) so a few hundred pages show server errors. The pages would load header and page of the page body but ...
4
votes
1answer
313 views

Weird URLs being access by Googlebot

Lately I've been seing all sorts of strange URLs show up as errors in my Webmaster tools account, but they're URLs that don't actually exist on my site, nor are linked from the pages that Google ...
1
vote
3answers
4k views

How to resolve “Google can't find your site's robots.txt” error?

I've recently found that "Google can't find your site's robots.txt" in crawl errors. When I tried Fetching as Google, I got result "SUCCESS", then I tried looking at crawl errors and it still shows ...