Strange entry in access log containing /RS=

Question

I have recently launched a client’s site and for the last few days and with increasing frequency I am seeing strange entries into the access log.

[28/Feb/2014:06:26:53 +0800] "GET //RS=^ADAA6U_G38x_VuWqDIVQJpBbDUsUW0- HTTP/1.1" 302 630 "http://www.domain.com.au/" "Mozilla/5.0 (Windows NT 6.0) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/27.0.1453.116 Safari/537.36"]

and

[28/Feb/2014:06:26:54 +0800] "GET /RS=%5eADAA6U_G38x_VuWqDIVQJpBbDUsUW0- HTTP/1.1" 404 8291 "http://www.domain.com.au/" "Mozilla/5.0 (Windows NT 6.0) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/27.0.1453.116 Safari/537.36"

The domain.com.au part is the actual address of the website. I have changed it for this post.

I cannot figure out what this is trying to archieve as it’s simply setting another get variable, which would not archieve anything, unless I am wrong.

1. Do you think we should be concerned with these requests?
2. What are these requests trying to archieve?
3. Anything we can do to stop them?

Answer 1

I couldn't find anything related to that request so if it is anything malicious it hasn't come to light yet. I also don't see how it could be used to harm your site.

Having said that, the bad guys seem to be one step ahead of us so if this request doesn't serve any useful purpose on your site I would try to block it. Even if only to keep your logs from being polluted.

I would say you should attempt to block them if possible. The two consistent parts of those request are /RS= and ADAA6U_G38x_VuWqDIVQJpBbDUsUW0 so they can be what you focus on to block them.

Answer 2

They come from web scrapers incorrectly using Yahoo! Search result. This discovery was made by @tenants at XenForo forum. They explain more of the implications of receiving these requests and how they handle them.

1. Do you think we should be concerned with these requests?

You don’t have to be concerned with these requests. They are just hallmarks of dumb bots and dumb bots wander all over the Internet. These requests are not to be identified as malicious just based on the URL, they are probably innocent.

2. What are these requests trying to archieve?

They’re trying to get the contents of the page they’re requesting. They have no special effect, they are (undesired) products of a Yahoo! Search scraping.

3. Anything we can do to stop them?

Not really, anyone is free to post whatever requests they like on the net. (At least technically. Social and legal aspects put aside.)

1. You can throw them away when generating reports from your logs. This is the option I chose.

2. Or you can try to fix the requests to succeed and not generate log entries. This is probably what most do from what I saw on the web. I see a flaw in this approach. While making their visitor’s experience better, they forget who those visitors are. Dumb bots. I don’t want dumb bots on my sites so I won’t bother to improve their experience.

If you want to fix the requests, you can do it using mod-rewrite, possibly called from .htaccess, e.g. using the code from the XenForo forum post I mentioned above:

RewriteEngine On

# strange behaving bots, these are urls scraped from yahoo (botters scrapping for links, yahoo search link contain RK RS) tenants modification:
RewriteRule ^(.*)RK=0/RS= /$1 [L,NC,R=301]
RewriteRule ^(.*)RS=^ /$1 [L,NC,R=301]

You may have to fiddle with the regexes a bit, e.g. add an additional slash after the (.*) if your URLs don’t end with one.

 Related

• original report of the nature or the requests @ XenForo forum
• RewriteRule for .html/RK=0/RS=[random_string] @ Stack Overflow
• .htaccess rewrite rule remove everything after RK=0/RS= @ Stack Overflow
• IT Q: ” .. //RK=0/RS=foo- ” in URI @ The Blackboard (Rankexploits.com)
• Web server logs containing RS=^ ? @ InfoSec Handlers Diary Blog (not very interesting)
• Added URI String RS=^ADA @ Graphicline (not very interesting)

Kudos to @dman’s answer on Stack Overflow and @webaware’s comment under this question for finding the XenForo forum post.