Tell me more ×
Webmasters Stack Exchange is a question and answer site for pro webmasters. It's 100% free, no registration required.
up vote 3 down vote favorite

The Yandex spider is a frequent visitor to one of the sites I manage. On occasion it replaces the page name with two ampersands and a space. So if the page is: 

/mypage.aspx?param=value

then it will try and crawl it as: 

/&& ?param=value

Any idea why it is doing this?

EDIT:
If I remember correctly the IP that this mistake is coming from is based in California and not Russia. I believe that they crawl US sites from a US based IP address. Not sure if that helps.

More information about the request: 

IP: 199.21.99.82
City: Palo Alto
State: California
Country: United States
ISP: Yandex Inc.
User-Agent: Mozilla/5.0 (compatible; YandexBot/3.0; +http://yandex.com/bots)
share|improve this question

1 Answer

up vote 0 down vote

Its probably mis-translating your URL, may be if your URL was more simplified such as mypage.com/page-two

share|improve this answer
3  
Do you have any backing information for this? There's absolutely nothing unusual about that example. The Yandex engineers would have to be pretty incompetent to screw it up. – Su' Aug 2 '12 at 19:15
help.yandex.com/webmaster/?id=1113851 It provides details as to what Yandex will and will not except as URL strings. – nycmixing Aug 2 '12 at 20:16
I agree with Su'. Also that links to their robot guidelines/instructions and has nothing to do with this. The link in the example is normal and valid. Yandex should not have problems with it. – Anthony Hatzopoulos Aug 3 '12 at 2:02

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.