I have a dynamic site which has many (well, less than 50) users. Each user is allowed to create as many pages as they want. I know that there is a limit to how many pages you can be listed in sitemap.xml, and for now I am under that limit, but I would like to know what the best way to enable search engines to know (all?) about my user's pages while still being under that limit even if I have more than the limit.
Note that you can specify multiple sitemaps in your robots.txt file:
If you're bumping into the limitation, perhaps divide your users up somehow. Let's say each of your users has no more than 10,000 urls a piece, you could break it up into files and add this to your
Now, you'd still have to be mindful of the 10MB limit for each individual sitemap, but this is an approach to handle the "too many" urls problem.