I have a problem with googlebot and bingbot accessing our web service. We have a search application created in aspx that is integrated into a drupal website. Users perform searches based on postal ...
I have an issue with bots crawling the mobile site and returning hits on the mobile site for desktop users. The mobile content is in /mobile/ The regular site will redirect mobile users to the ...
I've a system which blacklists users who request too many pages too fast if they are not in my white list. We are just worried about the main search engines and to be honest Google is the only one my ...