Tell me more ×
Webmasters Stack Exchange is a question and answer site for pro webmasters. It's 100% free, no registration required.
up vote 4 down vote favorite
1

So, I'm an 'avid' user of google webmaster tools and analytics. I'm building an e-commerce site, all with original product descriptions and a base of 'useful' industry-relevant content. I generated and submitted a new sitemap yesterday with some new content. Next to the sitemap in google webmaster tools, yesterday I had approx '600 urls in web index'. Today... 5. 5!?

There is no canocial issue, the site is configured properly to go to the non-www version, there are no 'spammy' links (probably no links at all at this stage...) and no duplicate or spammy content.

What has happened? To add further confusion, doing a site: search on google churns out 5.8k results, so it's still 'in the index', and monitoring certain keywords tells me that most of them still rank and rankings havent been changed... what is webmaster tools telling me by saying there are 5 urls in the web index?

share|improve this question
I have same issue I have submitted 3 urls in webmaster tools sitemap but only 2 are indexed and my home page is not indexed. – shahrukh Jan 28 at 13:45

2 Answers

up vote 3 down vote accepted

"5 URLs in web index" means that, of all of the pages in your current sitemap, only five of them appear in Google's index. This could be for a number of reasons:

  • You only have 5 URLs in your current sitemap. (If this is the case, '5 URLs in web index' is exactly what Webmaster Tools should be reporting.)

  • Some pages appear in your sitemap that Google hasn't crawled yet. Solution: wait.

  • Some pages appear in your sitemap that Google considers duplicate content. Solution: Check and fix duplicate content problems using a duplicate content tool. Check for duplicate page titles and descriptions in Webmaster Tools (under Diagnostics > HTML Suggestions). Check that all files which paginate, categorise, or present archives of content use the canonical tag.

  • Some pages appear in your sitemap as www.yoursite.com/page, but you're redirecting content to yoursite.com/page and dropping the www (or vice-versa). Google won't consider those pages to be in its web index because it sees URLs with 'www' as different from those without, even if the page itself is identical. [source]. Solution: If you're redirecting pages and stripping the 'www', make sure the pages are displayed without 'www' in your sitemap (and vice-versa).

  • Google can't crawl your sitemap or pages. Solution: Check and fix crawl errors (under Diagnostics > Crawl errors).

  • Google Webmaster Tools are having a bad day. (It happens.) Solution: wait.

The fact that the reported 'URLs in web index' figure dropped from 600 to 5 doesn't point to any one of these in particular, in my view, so it's worth running through all of them.

share|improve this answer
I'm still ranking for more than 5 products, so there are definitely more than 5 url's in web index. The 600 odd pages I had submitted have been crawled without a problem for the last month or two. There is no duplicate content on any of the pages, bar navigation and generic footer content. All url's redirect correctly to the non-www version of the site, as configured in the webmaster console. Sitemap uses the correct non-www version for all pages. There are no crawl errors. I think you may be onto something with Google Webmaster Tools having a bad day... Though to throw a further curveball – Anonymous Aug 23 '11 at 12:45
In Google Webmaster Tools on the www version of the site, it's still showing 400~ urls in the index. Though a site:www.mysite.com search brings no results. – Anonymous Aug 23 '11 at 12:46
@Chris It could well be a temporary hiccup. It's also worth noting that the "URLs in site index" figure doesn't represent the number of pages in Google's index, nor does it represent the number of pages you've submitted before. It simply represents the number of pages in Google's index that are also listed in the sitemap Google most recently crawled. 1. How many pages are listed in your current sitemap? 2. Are you using robots.txt to prevent crawling of any pages? 3. Have you made any other changes since seeing the drop? – Nick Aug 23 '11 at 14:08
Just over 600 pages in the sitemap, robots.txt is allowing full access to the site (google webmaster tools 'crawler access' tab confirms this). The only change I've made was adding a new page yesterday to a 'partners' page - links to manufacturer's of products we sells' sites, generated a new sitemap with the partners page and resubmitted via webmaster tools. Other than that, nothing... Hopefully it'll just be a glitch. – Anonymous Aug 23 '11 at 14:30
up vote 1 down vote

Probably a temporary issue. If your sitemap has been crawled correctly, your site still appears for a site: search and a keyword search, there is nothing to worry about.

There are many reason why there could be a discrepancy between the results from day to day. A few of theme are listed here on the Google Webmaster Central help forums:

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.