I am trying to create sitemaps for a very large multilingual website; means that every single URL is duplicated with as many languages there are; however the more pressing issue is that content is incredibly dynamic, the lastmod tag can be easily obtained.
The sitemap is composed as follows, each index contains and specifies every sitemap under it.
/sitemaps/index.xml /sitemaps/[language]/index.xml /sitemaps/[language]/[section]/[collection-timestamp].xml
If I create each collection point based on creation, the point is added, and hence a lastmod tag cannot be added or otherwise known other than by fetching the resource via a HEAD request and reading the header.
If I create each collection point based on modification, the point is added if it was modified during the day, and as so there will be duplicates entries between points with different lastmod dates, any data that changes; since it's impossible to modify already stored collection points as it will require intensive reads to modify data in older collection points.