network profile
| bio | website | |
|---|---|---|
| location | ||
| age | ||
| visits | member for | 7 months |
| seen | Jan 11 at 15:30 | |
| stats | profile views | 4 |
Want to learn everything but takes me a long time.
|
Jan 11 |
comment |
Force fresh content, not from the browser cache Thanks, I will try that |
|
Oct 5 |
comment |
Number of page requests by any Bot in 5 secs Actually I just pasted the test code on here. I would be using more refined codes for real implementation. I wanted this script only for bots without sessions as I have a number of other scripts that will protect my site for those conditions. Thanks for your help. This has been an educational thread. |
|
Oct 4 |
comment |
Number of page requests by any Bot in 5 secs According to developers.google.com/webmasters/control-crawl-index/docs/… 4xx (client errors) -
Google treats all 4xx errors in the same way and assumes that no valid robots.txt file exists. It is assumed that there are no restrictions. This is a "full allow" for crawling. Note: this includes 401 "Unauthorized" and 403 "Forbidden" HTTP result codes. |
|
Oct 3 |
comment |
Number of page requests by any Bot in 5 secs Thanks for taking the time to reply. The request table only has about max 80 rows at any given time. The script above has 800 set so that I want to trigger the condition only if there really is a need for it. I am hosting a site elsewhere and cant really play with the firewall. Thanks anyways. |
|
Oct 3 |
comment |
Number of page requests by any Bot in 5 secs Since I did not get any suggestions, I am setting it to 800 requests in 2 secs. This means my script will block all the bots if there are 800 requests or more in the past 2 secs. |
