We have a sitemap that is regenerated daily based on the records in our database. There are about 55 million records and each record is accessible as a separate page. However sometimes records are deleted and after 1 to 2 weeks the Google Search Console complains for a couple (but not all) of the deleted items that their URL returns a 404. This is shown as a warning under the console's Sitemap errors section with a link to the sitemap where this URL used to be, but is now removed.
I suspect the cause of this is that Google doesn't fetch the sitemap every day. It seems to cache it for a couple weeks, despite our http-response headers:
Cache-Control: no-cache, no-store, max-age=0, must-revalidate
Expires: 0
Google then complains when it checks cached URLs that have been removed both from the site and the latest version of the sitemap. Can someone confirm that Google always caches sitemaps?
Looking at the dates that the pieces of our sitemap are processed, it seems our entire sitemap also takes about 2 weeks to process. Is it possible to tell Google to fetch and use the latest version of a sitemap page daily?
I've read this similar question, but that question is asking where Google gets its old urls from. I know exactly where it comes from (Google told me so). I understand the 404s are probably not a big deal, but if possible I'd like to prevent them.