Is there a reason why a URL on our website would sometimes return 200 and sometimes return a 301? This appears to happen on a number of our URLs even something basic like example.com/robots.txt
When I load the URL myself - it always returns a 200
The site is configured to work as www.example.com/robot.txt - could it be that the bots are looking at the non-www version and getting redirected?
When access logs show differnt behavior for the "same" URL, make sure it's indeed the same url: check the protocol, the subdomain, the TLD, zones, path and the query string. Default access log logging pattern is pretty bad and misses a lot of dimensions.
Google does not normally use sitemap for crawling and when it does, it's just a supplemental reference. Main reference would be links found on the web. So you can easily get google crawling urls that are different from what's in sitemap. You have rel canonical and redirections logic to mend that.
Here how access log works with redirections:
Client (Googlebot in our case) sends a request url (say, with "wrong" subdomain or protocol) to the server to port :80.
On the server, on port 80, we have a web-server software that listens for requests like that. It gets the request and runs the request through a bunch of logic to understand what to do to it. It sees that it has rules (typically in .htaccess, but not always) to redirect all http:// urls to their https:// version, so it sends a response to the client with code 301 and location field set to the https:// version of the url.
Right after it sends the client away, without waiting for the next hit, the web-server makes a record in its access logs indicating that there had been a hit and it dealt with it so and so. After it, the web-server forgets about that client forever.
The client gets the response from the server and then it organically makes another request to the location of the received response, so to the https:// version of the page.
The web-server gets another request and treats it as if there were never the original request: it goes through a bunch of logic and sees that this pattern of request has to be given away to php to get a response, so it gives the request (and the cookies) to the php endpoint and php returns a body to respond to the client. The web-server responds to the client and makes another record in the access log.
Thus, you will almost always get another hit in logs after a 301 hit.
I'm missing a lot of nuances here, like CDNs, SSL/TLS handshakes, http2 handshakes, but this is just to give the very basic understanding of how web-server logs work.
