Google sitemap for dynamic url structure

Question

I am wondering what the best approach would be to generating a sitemap for google to crawl with a highly dynamic url structure.

We have a webapp that assigns custom uris to all the users in our database for direct access to their content. All the uris are namespaced to one endpoint. The uris are chosen by the user who owns the profile and assume a human-readable form ('/profile/someguy')

Would google require that all possible uris be documented in the sitemap? Wouldn't this lead to a ridiculously large xml file? It would be possible to periodically generate this using a script and break it up into smaller sitemaps if that is what's necessary.

I have searched the webmaster docs for answers to this, but they largely only cover dynamic urls that use query strings, not ones that are generated in a readable format.

Any guidance is much appreciated!

Answer 1

You should have a script coded that will query your database and re-generate your XML sitemap file say every few days. Set a cronjob to execute the script, and have it re-submitted to Google and Bing using their API. This way it's always up to date, and the search engines are always getting a fresh copy.

Answer 2

Include every URL you want to notify Google about in your sitemap. Sitemaps can be very large in size (50,000 URLs) but if that isn't enough you can sitemap index files to link multiple sitemaps together allowing for an unlimited amount of URLs to be included in your sitemaps.