How do I replace all my XML Sitemaps without hurting SEO?

Question

I am looking for someone to tell me the appropriate procedure to replace all Google sitemaps. I can clearly go in and delete the old ones and simply replace them, but as it can sometimes take many days to get the new sitemaps indexed, will this cause a drop off in my indexing.

Is it best to add the newer sitemaps, leave the old ones and once the new ones have been indexed, to delete the old ones, or is this bad for SEO?

Additionally, I am dealing with a directory site that has around 500K+ pages. Is it acceptable to have a sitemap index file that links to all of the individual sitemaps. The index files as well as the individual sitemaps are all gziped. Is there any problem with this setup?

Answer 1

Sitemaps have very little effect on SEO. See The Sitemap Paradox. Deleting your old sitemaps shouldn't be an issue. Google will continue to crawl and index the URLs they contain even when the sitemaps are no longer there. In fact, getting Google to forget about URLs is pretty much impossible. If you have new sitemap files that list the URLs you will still be able to submit them to Webmaster Tools and see the stats about your pages that are only available when URLs are in a sitemap.

Each sitemap can only contain 50,000 URLs. Splitting up your 500K URLs into multiple sitemaps is your only option. gzipping all your sitemaps is absolutely fine. Even the index sitemap can be gzipped.

Answer 2

Definitely gzip sitemaps because that lowers downloading time, thereby making the server faster more often. Let google do the decompression at their end for you. If timing is an issue, what you want to do is click the gear icon and visit "site settings" in webmaster tools for all websites you have that the sitemap is taking forever to process, then for the crawl rate, choose the maximum number available. Usually google will allow you to set it so google crawls no more than two pages a second, but if google likes your content, then in a couple of days or so, they will let you increase the crawl rate beyond two pages.

Then mathematically speaking, provided google never does anything else but checks your pages (which I doubt), The time you need for google to scan every file in your sitemap is:

500,000 pages divided by two pages a second 
= 250,000 seconds 
or 4,166.6 minutes
or 69.4 hours 
or roughly three days.

What I suggest since you're on the right track so far is to either wait at least 30-60 days for google to finish scanning every page in your sitemap or keep checking the site settings and pray you can increase the crawl rate. I had it at one point where I could have google crawl my site containing about 1 million links at up to a rate of 10 requests per second.

Just make sure the content of every page in your site map is of high quality as defined in google's webmaster guidelines or your SEO will hurt.