This message has nothing to do with the sitemap itself. It has to do with the response times of some of the pages listed within the sitemap. So resubmitting a sitemap does nothing for you.
I have gotten these messages before and was deeply concerned and tested my system from within the network and externally. I tested the pages in question and a whole host of other pages trying to figure out what was wrong.
I finally came to the conclusion that it had nothing to do with my site at all. Part of this time measure is a whole host of elements that all have to work perfectly and often does. However, slow DNS response times, poor routing, and so on can effect Google's ability to download your pages. But let us not discount that quite possibly your server was slow at the moment.
But then I remembered back to Google's own page speed tests that I conducted and how much higher they were when comparing them side-by-side with other tools. I came to the conclusion that no matter how massive the bandwidth Google may possess, there are too many elements between them and my site and perhaps my site was a bit slow, but Google over-reacted and that I would give it some time.
Sure enough. It was a temporary thing. Nothing to worry about unless systemic. I have not seen these messages again.