Take the 2-minute tour ×
Webmasters Stack Exchange is a question and answer site for pro webmasters. It's 100% free, no registration required.

I have a sitemap on my site at this location:

https://www.loveelectronics.co.uk/handlers/sitemap.ashx

which is referenced in my robots.txt like so:

Sitemap: https://www.loveelectronics.co.uk/Handlers/Sitemap.ashx

For some reason, I keep getting requests for the following files:

sitemap.xml
sitemap-b.xml
sitemap-g.xml
sitemap-r.xml

What does this mean? Why are agents requesting /sitemap.xml when i specify a different path in my robots.txt, and why are they requesting these odd sitemap-?.xml files. I am getting many requests for these files a day, and I'd like to get this cleared up.

share|improve this question
add comment

2 Answers

You're probably just dealing with a relatively dumb spider.

/sitemap.xml is the "standard" location, which explains that request. Whatever spider/bot is hitting that location is just taking the safe bet. Hopefully, for them anyway, they're also looking at your robots file.

As for the others, they're not standard names, but most likely just something the spider encountered on another site and stashed for future attempts, eg. Lyricspedia's sitemap index which has some of those same filenames, minus the .gz. It's possible it was just programmed with a few such patterns, since this one specifically seems kind of likely.

share|improve this answer
add comment

Well, maybe it is trying to see if multiple XML sitemaps may have been created/uploaded (e.g. in the case of XML sitemaps that have been split into multiple because each hit the upper limit of max URLS per XML sitemap file)

share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.