Tell me more ×
Webmasters Stack Exchange is a question and answer site for pro webmasters. It's 100% free, no registration required.
up vote 3 down vote favorite

I consider using IIS Dynamic IP Restrictions Module in order to make it hard for DoS (Denial of Service) attacks. But I am not sure how that would work out for search engine bots like Googlebot.

Any idea?

share|improve this question
It may consider numerous random connections from Google as a DoS since they are all direct connections and are not going from link to link with referrals being your own internal links. If you haven't been DoSed I wouldn't install this. I rather have traffic than chance blocking bots and visitors. Rather than blocking IP based on conditions you should look for applications that monitor packets. – Anagio Feb 28 '12 at 9:26
@Anagio do you know that if it's possible to allow bots by reverse DNS lookup with IIS Dynamic IP Restrictions Module just like it explained here: googlewebmastercentral.blogspot.com/2006/09/… support.google.com/webmasters/bin/… Any idea? – tugberk Feb 28 '12 at 9:36
1  
It's possible but that is quite a bit of work and bandwidth for a server to reverse DNS every single hit it receives. I personally wouldn't use reverse DNS like that. Here is one resource I can provide, I suggest Googling for applications to monitor packets if possible slideshare.net/robertosl81/… – Anagio Feb 28 '12 at 9:43
Another resource technet.microsoft.com/en-us/library/cc750213.aspx "Recommended Registry key configuration to thwart DoS attacks" – Anagio Feb 28 '12 at 9:44
1  
Anyone solve this? You can allow specific IP addresses in the dynamic ip restrictions module, but google don't publish a 'supported' list of IP's because they change. If you could allow traffic based on user agents that would work, but you currently can't in the module. – Matthew Evans Sep 7 '12 at 9:16

1 Answer

up vote 2 down vote

In the comments is was mentioned that this could be done by whitelisting google's user agent. This is a bad idea: lots of bots can impersonate it, and then will be able to dos your site.

Looking at the feature list of this script, it seems that it uses two criteria to determine whether it's a dos attack:

  • The number of concurrent requests.
  • The number of requests over a period of time.

For google, you can easily limit both of those things to below the threshold that the script uses by setting a custom crawl rate in google webmaster tools. This will mean that google will not crawl your site as often, but they wont overload your sever.

I'm not sure what you could do about other search engines, besides making sure that the thresholds for your script are higher than the crawl speeds that they use.

share|improve this answer
really nice answer. Thanks! – tugberk Sep 14 '12 at 7:16
@tugberk if this answer works for you, then could you accept it (click the check mark next to the voting arrows) so that people know you solved your problem. Thanks. – Christofian Sep 14 '12 at 22:07

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.