I have a sitemap on my site at this location:


which is referenced in my robots.txt like so:

Sitemap: https://www.loveelectronics.co.uk/Handlers/Sitemap.ashx

For some reason, I keep getting requests for the following files:


What does this mean? Why are agents requesting /sitemap.xml when i specify a different path in my robots.txt, and why are they requesting these odd sitemap-?.xml files. I am getting many requests for these files a day, and I'd like to get this cleared up.


You're probably just dealing with a relatively dumb spider.

/sitemap.xml is the "standard" location, which explains that request. Whatever spider/bot is hitting that location is just taking the safe bet. Hopefully, for them anyway, they're also looking at your robots file.

As for the others, they're not standard names, but most likely just something the spider encountered on another site and stashed for future attempts, eg. Lyricspedia's sitemap index which has some of those same filenames, minus the .gz. It's possible it was just programmed with a few such patterns, since this one specifically seems kind of likely.

| improve this answer | |

Well, maybe it is trying to see if multiple XML sitemaps may have been created/uploaded (e.g. in the case of XML sitemaps that have been split into multiple because each hit the upper limit of max URLS per XML sitemap file)

| improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.