New answers tagged indexing
It turned out to be a bug in Google image search I think. There by I changed the site script to checked the HTTP_USER_AGENT and if it is not Google or Yahoo or msnbot it will load in Ajax even if the _escaped_fragment_ is detected in the url. This is a sad solution. not going to mark it as the correct one.
I had a look at your XMLs as well as some sections of the site, to answer your questions: there is no fixed time for the sitemaps to get indexed, it depends on a whole bunch of factors like size of your site, crawl friendliness of your site (more on this later), interna/external links to the site etc. Now it actually does not matter if you list all URLs in ...
The sitemaps with errors, I guess those speak for themselfs. To answer the remainder of your question: Yes it is normal that they dont index all. This number will slowly increase in time, but cán stop before it'll reach 100%. That means they find some pages not worth their while. That could be because of duplicate content, bad content/title/description, ...
You can't predict when Googlebot goes to your site for indexing. Googlebot follows links it finds over the Internet and it can find a link to your site (and goes to your new article) right away after publishing. Googlebot may also be not going to your site for several hours, days, etc. It's the way Googlebot works. To optimize a quick indexing, you can: ...
No difference at all as far as crawlers are able to access them (with all your valid rules) with HTTP 200 response.
There is no difference to search engines between those two methods. You can generate it like you want. Moreover, you need to know blocking a page with robots.txt is not the perfect way to prevent indexing of pages. You can read this question for more information.
The graph you are looking at in Webmaster Tools showing 'removed' I presume that is the index status graph and if so, that shows URLs removed by manual request and not through normal 404 site maintenance. Look under Crawl > Crawl Errors and you should see recent 404 pages.
Days, Weeks and Months... It can take Google days, weeks and even months for Google to remove pages marked noindex, robots and 404's. Generally it takes Google several crawls before Google acts on the new information of a page. User Errors It should be also noted that more than often users make human errors and create 404's, noindexs and so forth by ...
Sadly, you are taking all of the steps with the removal process. Google Webmasters also explains this issue with their YouTube video. It takes a good while for the actual removal process from all of Google and the search engine. Try to be patient now, and it will happen. Also, you have to remember, Google is indexing, re-indexing, and deleting records from ...
If you marked those pages with a noindex meta tag, there are two possible reasons I see: Googlebot didn't reindex the site yet and you don't have wait enough (it can take a lot of time like several weeks, it depends on the site crawl rate) You use 404 instead of 410 HTTP status (Gone)
Top 50 recent answers are included