Tagged Questions

2
votes
1answer
41 views

“Google couldn't access your site because of a server connectivity issue”

In Google Webmaster Tools on my dashboard I have a notice that appeared 2-3 days ago that says "Googlebot can't access your site". Under the Current Status it has: DNS - SUCCESS Server Connectivity ...
0
votes
1answer
38 views

How to overcome duplicate URLS

I am having a website which is 3 years old, now I wanted to do some SEO chages into it. so, changed the some urls. From: /psd-layouts.html to /psd-web-layouts.html Now my question is that, I had ...
1
vote
2answers
106 views

Will 301 redirection to a Google Indexed URL can remove the URL from Google index?

I need to remove some indexed URLs from Google SERP. The problem is I cannot use the 3 types of removal patterns Google suggests like robots.txt, meta name "noindex" and remove the page and make it as ...
1
vote
3answers
100 views

Google Indexing Pages with noindex meta tag

I got a message in Google Webmaster Tools about "Googlebot found an extremely high number of URLs on your site" with a long list of example URLs. For some of the pages listed, there was nothing to ...
1
vote
3answers
132 views

URL blocked by robots.txt in webmaster for blogger site

Remove: Disallow: /search From where can i remove this line. I am using blogger and in webmaster it shows my many pages are blocked. From one of your answers i know that i need to remove this text ...
3
votes
1answer
71 views

Need help with 200,000 + 404's

The business I work for recently migrated our e-commerce site from Celerant to Magento. Despite using a robots.txt file and a 301 redirect extension, webmaster tools is reporting that our "not found" ...
1
vote
2answers
128 views

Google Webmaster Tools complains about missing robots.txt

Nine days ago, I got a message Google Webmaster Tools: Over the last 24 hours, Googlebot encountered 1 errors while attempting to access your robots.txt. Well, but I don't have a robots.txt on ...
2
votes
2answers
208 views

Network unreachable: robots.txt unreachable

I am trying to add a valid sitemap to Google Webmaster. Yet, it says: Network unreachable: robots.txt unreachableWe were unable to crawl your Sitemap because we found a robots.txt file at ...
1
vote
3answers
501 views

Robots.txt never downloaded but some blocked URLs in GWT

There is something I don't understand in Google Webmaster Tools (GWT) for my Wordpress site. In menu "Blocked URLs", it mention that my robots.txt has never been downloaded but there are some blocked ...
0
votes
1answer
216 views

google webmaster showing warnings - blocked by robots.txt, but its not?

im getting a warning when submitting my sitemap to webmaster tools it saying that my pages are blocked by robots.txt but its not, has any one come across this or a way to resolve this before ? the ...
1
vote
2answers
626 views

How to resolve “Google can't find your site's robots.txt” error?

I've recently found that "Google can't find your site's robots.txt" in crawl errors. When I tried Fetching as Google, I got result "SUCCESS", then I tried looking at crawl errors and it still shows ...
6
votes
4answers
311 views

robots.txt not updated

I have updated some url's and files in robots.txt file to block url's and files from google search results but, still files displaying in search results. As per a suggestion from a site I tried to ...
4
votes
1answer
2k views

Google Webmaster Central tells me that robots is blocking access to the sitemap

This is my robots.txt User-agent: * Disallow: /wp-admin/ Disallow: /wp-includes/ Sitemap: http://www.mydomain.org/sitemap.xml.gz But Google Webmaster Central tells me that robots is blocking ...
1
vote
2answers
260 views

How to submit sitemap when your website has partial https? - Error: “Not in Domain”

My website is an eCommerce that is set up to do HTTP for the item browsing portion, but HTTPS for things like shopping cart, contact us, etc.. (anything that has forms on it). I've submitted my ...
2
votes
3answers
4k views

How to remove old robots.txt from google as old file block the whole site

I have a website which still shows old robots.txt in the google webmaster tools. User-agent: * Disallow: / Which is blocking Googlebot. I have removed old file updated new robots.txt file with ...
2
votes
1answer
173 views

What methods are effective/necessary to quickly remove URLs from the Google index? [duplicate]

Possible Duplicate: How to Remove URLs from Google Search Engine In short, I have ~1000 URL's that need removing from the Google index ASAP, this is as of a week or so ago, and over that ...
3
votes
2answers
5k views

Can I invoke Google to check my robots.txt?

I read the answers in this question, but they still leave my question open: does Google cache robots.txt? I didn't find a way in the Google Webmaster Tools to invoke a re-download of my robots.txt. ...
2
votes
1answer
393 views

Why am I getting this message “Some important page is blocked by robots.txt”?

My site's URL is: www.hackinguniverse.org From some day in Google Webmaster tool a message is showing that says "Some important page is blocked by robots.txt". My robots.txt is: User-agent: ...
1
vote
1answer
525 views

“Restricted by Robots.txt” in Google Webmaster Tools when no robots.txt exists

In Google Webmaster Tools, I have 832 errors in the sitemap with the error being "URL restricted by robots.txt". I don't have a robots.txt file, and the robots meta tag equals "index, follow". I did ...
4
votes
3answers
553 views

nofollow doesn't stop crawling?

In my website, most of the pages has a linkedin sharing button written in this way: <li> <a id="share_linkedin" ...
1
vote
1answer
1k views

Google Webmasters Tools: Denied by robots.txt

I have a website: http://46.105.30.3/ And I'm trying to get testing it using the Fetch as Googlebot option in the Google Webmasters Tools. When I try it it returns Denied by robots.txt. My ...
2
votes
2answers
4k views

Googlebot crawler: URL restricted by robots.txt - how to fix the entries in robots.txt?

I noticed recently that Google is not caching all of the pages on my website. Upon using the Google webmaster diagnostic tool, I realized that some of my pages were being restricted by entries in my ...