All Questions
4 questions
1
vote
1
answer
277
views
Calculating requests per second within bing
In Google Search Console, I am able to set the rate at which google crawls my site and it tells me the details such as the requests per second and seconds between requests. In Bing Webmaster tools, I ...
- 13.6k
5
votes
1
answer
783
views
Am I covering the major search engine spiders in my anti-crawl protection white list?
I've a system which blacklists users who request too many pages too fast if they are not in my white list. We are just worried about the main search engines and to be honest Google is the only one my ...
- 421
3
votes
2
answers
2k
views
Do all search engines obey nofollow rel attribute on hyperlinks
I have been informed that search engine support is inconsistent for the rel attribute with the 'nofollow' value.
Is this the case.
From my research Google, Yahoo and Bing seem to obey this attribute ...
- 145
1
vote
1
answer
684
views
How does bing-bot( is that the right spider-name? ) and googlebot interpret 301 redirect?
I've been looking for documentation on how the Microsoft and Google bots interpret 301 redirects.
It seems that google-bot stores documents on a url based index system. But I haven't been able to ...
- 185