Webmasters Stack Exchange is a question and answer site for pro webmasters. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I have an e-commerce website www.beyondtime.in. I have been constantly monitoring Googlebot crawling on my website and my webmaster account. Lately, I have found two issues that I have not been able to understand.

1.) The Google Bots have been only crawling www.beyondtime.in/telecom.php when the URL is not even valid. What needs to be done to let Google crawl other pages of the website as well?

2.) The second question is about the Google Webmaster account, where I've submitted my sitemap with 227 URLs. Out of that, only 156 have been indexed. None of the images of my website have been indexed by Google.

share|improve this question
You say that googlebot has only crawled that one page, yet you have many indexed pages - so googlebot must have crawled many pages? How did you measure this? – w3dk Sep 27 '12 at 14:15
w3d.. we have the log system of our site.. who-ever visits our page, it will store the IP and the Name of the system..the name is always google-bot – Anusha Sep 28 '12 at 3:53
up vote 3 down vote accepted

Your images are probably not indexed because you block the bot in your robots.txt file to crawl directory /images.

At first glance I would also add a User-agent: * line above the Allow: / line.

Also add the line

Sitemap: http://www.beyondtime.in 

to your robots.txt file (to let more search engines detect it).

Double check your webmastertools (under section Status > Blocked Urls) say nothing is blocked.

You have a low (a very low) number of backlinks so far (as verified by Opensiteexplorer. That and in combination with low traffic can lead to slow indexing of all your pages. A quick check with this SEO spider tool results in good looking status codes.

One BIG Problem I see is that almost every page has a rel=canonical directive linking to the homepage. That reads like "Almost every page is a duplicate of the homepage, so ignore it (the page with the canonical-reference).

(If this is all fixed I would double check if my sitemap.xml lists all pages and if the pages allow indexing and following).

share|improve this answer
thanks for your reply.. most help-ful.. – Anusha Sep 27 '12 at 12:17
But why The Google Bots have been only crawling beyondtime.in/telecom.php this URL of my website, when the URL is not even valid. – Anusha Sep 27 '12 at 12:22
The link could have existed at some point in time. Another reason is: In order for your site to get noticed by a search engine someone placed a link to that url, or that url was submitted to a search engine, things like that. – initall Sep 27 '12 at 12:40
Just to add, XML sitemaps are only a helpful suggestion for Google. There is no guarantee that Google will index all the pages in your sitemap. – w3dk Sep 27 '12 at 14:19
@w3d Correct. But invented especially to allow to make the search engines do a better job and potentially crawl ALL pages (sitemaps.org). It's a helper, a fallback thing for Anusha if some of his pages are not interlinked. – initall Sep 27 '12 at 14:31

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.