Take the 2-minute tour ×
Webmasters Stack Exchange is a question and answer site for pro webmasters. It's 100% free, no registration required.

When analyzing traffic to some of my sites, I almost always notice a large amount of traffic from obscure sources like Japan, Russia, and other countries that really have no business visiting my site(s).

Most of the time, these "visitors" are viewing the site(s) considerably more than other visitors that appear to be legitimate.

Are these bots that index my site(s) or are they malicious by nature and should be banned using an IP deny filter?

Also, is there a better way to handle these visitors so that I can both determine their intention(s) and "herd" them away from my site(s)?


EDIT:

If I can't determine the intention(s) of these "visitors" by using all available resources, would it be a bad idea (in terms of SEO) to create a page/section of my site(s) that, if visited, would tell me without a doubt that the "visitors" are bots (not robots, but bots that pry the site to expose flaws)?

share|improve this question
    
Can you provide some IP addresses and domain names? I will check my database to see if I can tell you something about them. –  closetnoc Mar 9 at 0:08

2 Answers 2

Are these bots that index my site(s) or are they malicious by nature and should be banned using an IP deny filter?

You tell us. If they're interacting with your site as any normal user would there is no need to block them. If they're using the site more than visitors from other regions then your site may appeal to them then users from other regions.

Also, is there a better way to handle these visitors so that I can both determine their intention(s) and "herd" them away from my site(s)?

Before you go blocking IP addresses, or entire ranges of IP addresses, you need to be sure these users are malicious. Check your logs. Do you see malicious requests? Do you see unusual activity beyond longer visit durations and page requests? What user-agents are they using? Do they identify themselves as bots? If so, have you researched them? Do they honor robots.txt?

Before you can determine a course of action you need to determine if there is even an issue here. As of right now there isn;t anything suspicious or unusual about what you've posted here.

If I can't determine the intention(s) of these "visitors" by using all available resources, would it be a bad idea (in terms of SEO) to create a page/section of my site(s) that, if visited, would tell me without a doubt that the "visitors" are bots (not robots, but bots that pry the site to expose flaws)?

Sure. Just put a link to a page on your website that humans can't find and is blocked by robots.txt. This will keep humans and good bots out. Then when this page is hit by a bad bot you can log it. Because it did not obey your robots.txt file you can lean towards it being a bad bot and blocking it. You can also compare it to your logs to see what it is up to to make a more informed decision as to your course of action.

share|improve this answer

You could try something like Google analytics (http://www.google.com/analytics/) that will let you see what they are doing. You also might want to consider why they have no business on your site, and consider expanding to cover what could be an interested community. also, not everyone it countries like china are malicious and wan't to steal stuff.

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.