google removed links from the index [duplicate]

Question

I asked google to crawl a site through webmaster tool and include it to the index. How actually it was done:

Firstly, i picked some urls manually. Approx. after half of a day those pages were successfully added to the google.

Then i asked him about another urls and chose one page (mainpage of foreign branch of the website) as a start point of crawling and indexing the whole website. After sometime the pages which were asked to be crawled manually were added successfully to the index, but google didn't add all pages, though i asked him to do this.

I check the site through the seo-browser.com service and didn't find any glitches with linking pages etc. and didn't get any bad warning messages from google

How does google actually treat manual request to crawl and index website? Would please someone recommed authoritative guide on this issue or provide with some advice?

And another one interesting trick by google! Some of those pages which i had submitted manually at first time and google had added them to the index were removed from google index today, though they still in a a cache.

What reasons entice google to remove links that previouslly were added in there???

It seems like google doesn't like my website at all)))

Answer 1

When you submit a site to Google you get the following message:

Google adds new sites to our index, and updates existing ones, every time we crawl the web. If you have a new URL, tell us about it here. We don't add all submitted URLs to our index, and we can't make predictions or guarantees about when or if submitted URLs will appear in our index.

Basically even though you have submitted your site and perhaps even a sitemap too Google doesn't mean it will process a complete index on the first visit, in fact it can take many crawls and depending on the amount of pages you have can take days, weeks, or months.

Depending how important and fresh Google believes you site is the more often the Google bot will return to your site and freshen its own index. Google has limited resources and their are over a billion pages updated daily and a lot of these pages and sites have priority over yours until your site has matured and again until it believes the importance and freshness of your site.

De-indexing of pages

Pages that lose their indexing can be for a few reasons, some new sites get a sandbox effect and this can take weeks to months to get out off, Again this can be a great time to keep your pages fresh and keep adding content. Time, Content and Backlinks are a great way of getting out of the sandbox effect. Also a recent edit to robots.txt can cause pages to drop out of index and remain cache, as well as using meta tag noindex on the page.

Also sometimes I've seen pages end up in the index and then a few hours later vanish and then the following day return. This could be the case assuming you have waited over 24-48 hrs.

Patience

This type of question gets asked weekly and your see just how many people report the same issue if you using the search function on this site. Your are in the same boat as a lot of people and keep doing what your doing 'Adding Content' and checking webmaster tools for crawl errors, the rest is out of your hands - just be patient.