2

I have a sitemap on my site at this location:

https://www.loveelectronics.co.uk/handlers/sitemap.ashx

which is referenced in my robots.txt like so:

Sitemap: https://www.loveelectronics.co.uk/Handlers/Sitemap.ashx

For some reason, I keep getting requests for the following files:

sitemap.xml
sitemap-b.xml
sitemap-g.xml
sitemap-r.xml

What does this mean? Why are agents requesting /sitemap.xml when i specify a different path in my robots.txt, and why are they requesting these odd sitemap-?.xml files. I am getting many requests for these files a day, and I'd like to get this cleared up.

1

You're probably just dealing with a relatively dumb spider.

/sitemap.xml is the "standard" location, which explains that request. Whatever spider/bot is hitting that location is just taking the safe bet. Hopefully, for them anyway, they're also looking at your robots file.

As for the others, they're not standard names, but most likely just something the spider encountered on another site and stashed for future attempts, eg. Lyricspedia's sitemap index which has some of those same filenames, minus the .gz. It's possible it was just programmed with a few such patterns, since this one specifically seems kind of likely.

0

Well, maybe it is trying to see if multiple XML sitemaps may have been created/uploaded (e.g. in the case of XML sitemaps that have been split into multiple because each hit the upper limit of max URLS per XML sitemap file)

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.