I've structure my video site like this:
/ (root) index.html -/videos -/video-id1 (dynamically generated subdirectory when user uploads a video) /index.html (dynamically generated file when user uploads a video)
The subdirectories and files in them are created dynamically each time a user uploads a video. I thought using a
sitemap.xml would be a good choice to get those files indexed in search engines because using PHP I can edit the sitemap dynamically after each page is created. But this is quite hard to do (since it's an XML document).
So I thought of using a
robots.txt instead of a sitemap to get them indexed, but would that be helpful at all for this?
In other words, I'm trying figure out how I can submit dynamically created subdirectories and files to search engines? Using Stack Exchange as another example, how do questions like this one get listed in search engines every time a user asks a questions?
I think they use
sitemap.xml for every question, as can be seen here.