3

I'm seeing error logs on a website because something tried to access:

example.com/&wd=test

the HTTP_REFERER is http://www.baidu.com/s?wd=FQQ

the error is from ASP.net:

A potentially dangerous Request.Path value was detected from the client (&)

The & char is not allowed in that position, it is allowed only after the ? char.

I'm wondering why are these hits happening. Is this a Baidu feature or is it bad bots?


UPDATE: I checked some of the ips using https://www.abuseipdb.com and I see other websites are reporting these ips as web attacks,

here are examples:

https://www.abuseipdb.com/check/111.206.36.143 https://www.abuseipdb.com/check/111.206.36.13

most reports about the &wd=test but there is some other stuff too

  • If I'm not mistaken, baidu is something from china. a search engine I think. But there's no point in trying to block bots since they will come back time and time again. If you want to, you can try to customize the error pages so the bots don't think they're successful so much. – Mike Feb 12 '18 at 19:27
2

This is a request from a kind of Baidu searchbot. Baidu is a search engine from China. Like Google has its own searchbot, Googlebot, so Baidu has its own. There is nothing suspicious and dangerous on this request.

If you don't like it, from statistical point of view, you can block it with your robots.txt, like

#Baiduspider
User-agent: Baiduspider
Disallow: /

Or block it with server configuration, like on Apache with:

<IfModule mod_rewrite.c>
    RewriteCond %{HTTP_USER_AGENT} baidu [NC]
    RewriteRule .* - [F,L]
</IfModule>
  • i doubt that it's a bot from baidu, why would they do such a request, the response for this particular malformed url is empty string (nothing) – Omu Feb 14 '18 at 11:27
  • As could happen, that your site was found in Baidu for any searchterm. And if somebody klicked on your site in Baidu SERPs, this click got the referrer from Baidu. – Evgeniy Feb 14 '18 at 11:39
1

It's most likely Baidu just being weird, and their bots sending traffic to invalid URLs.

If Baidu is sending you traffic, even to weird parameters such as that I wouldn't necessarily start blocking their bot. But you block them if you want in your robots.txt and because Baidu is a legitimate crawler, they should most likely respect your robots.txt file. Baidu is a major Chinese search engine.

The invalid URLs that they are sending traffic to shouldn't cause any issues for your site unless there is something on your server that causes errors in this regard. Most likely, the visitor will just be sent an invalid page.

You can remove these ? parameters in your htaccess file and 301 redirect back to the correct page. If you want to do that, simply run a Google search for "remove everything after htaccess stackoverflow". Remove every character after the question mark, and then have htaccess simply remove the question mark itself. This will keep the user on the page that was intended.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.