All Questions
Tagged with web-crawlers spam
12 questions
5
votes
2
answers
4k
views
Grapeshot crawler ignoring robots.txt
Has anyone come across a crawler called Grapeshot? They are hammering the same page repeatedly on our website. I believe they are looking for ad related keywords, based on previous content ad ...
- 385
2
votes
2
answers
1k
views
Is it okay to block all user agents that start with "PHP/5."?
I'm getting a lot of spambots from China (and elsewhere, likely botnets) accessing my blog which have a user agent string starting with PHP/5.. These include such nonsense values as PHP/5.3.70 and PHP/...
- 200
2
votes
1
answer
793
views
htaccess 410 url spam hack requests excluding site generated request
Our site was hacked and links to random content where added to the site. We completely removed the hacked site and put a new one in its place. Everything new including images and content, no other ...
- 29
2
votes
0
answers
43
views
One day after putting my site online and registering it with Google I got spam mail. How?
I put my website online yesterday and registered it with Google at the same time. Before that, I didn't even put a single link to the site anywhere online. And I don't have any email address on my ...
2
votes
0
answers
568
views
What % can be regarded as "normal" "not viewed traffic" reported in AWStats?
Recently statistics for my site is about 50% "not viewed traffic". This seems much as many robots are blocked. What % of "not viewed traffic" can be regarded as "normal"?
Is my site under attack when ...
- 71
1
vote
1
answer
627
views
How to block a spam bot .htaccess proper formatting?
I have downloaded my access log file from my cPanel and found I have a bot I would like to stop. In the access log, there User-Agent string reads the following;
88.80.205.219 - - [08/May/2014:23:29:...
- 11
1
vote
2
answers
250
views
Excluding Google Analytics spam by referral [duplicate]
I've been reading up on how to filter out spam from Google Analytics. In this case, I'm trying to filter out the spam that shows up under referrals.
In the Filter Pattern field of Analytics, I have ...
user47053
1
vote
1
answer
115
views
Commission Junction links are registering tons of (probably false) clicks
I have a sidebar of commission junction ads, and they are plain text links (a-tags) with rel="nofollow" attributes in them
I'm getting hundreds of "clicks" per day in my daily stats on CJ's dashboard....
- 111
1
vote
0
answers
45
views
How do spambots find and submit to email opt-in the end of a funnel?
I have an opt-in page that's being nuked right now with spambots. A simple webpage that I use to allow subscribers into my email list.
The only way to get to that page was through one of my tracking ...
- 11
1
vote
0
answers
18
views
Google displaying wrong site title and snippet [duplicate]
I am having an issue where a website that displays the incorrect site title and snippet in Google Search.
From the content it is clear that some spammer has infected the site, but I cannot figure out ...
- 11
0
votes
1
answer
668
views
Is it a good practice to block spam bots by IP ranges?
I'm getting too many hits on my website from Yandex. It doesn't obey robots.txt and by some way it can bypass .htaccess rules. So I'm thinking to block all Yandex IP ranges in my system firewall.
Is ...
- 257
0
votes
0
answers
15
views
spam referers bypass htaccess rules [duplicate]
Here is a sample code I'm using in .htaccess to block spam referers:
RewriteEngine On
RewriteCond %{HTTP_REFERER} ^http://.*buttons-for-your-website\.com/ [NC,OR]
RewriteCond %{HTTP_REFERER} ^http://....
- 257