We have a self generating honeypot that is designed to fill crawlers with junk data (among other things). Its protected with the right "treaties" and headers so 99% of good bots stay away. Today ...
I know that nofollow prevents link juice from being passed across a link. But if search engine robots discover a link with a nofollow on it, will they add that link to their crawl queue? In other ...
I have pages with noindex in robots meta tag (specifically inner site search results pages). Because my site has many pages and Google hasn't crawled them all and I'm trying to help his crawler focus ...
We have a product navigation engine that allow our customers to filter products based on attributes (size, color, etc.). We are implementing an upgrade that allows them to use "multi-select" ...
I have been informed that search engine support is inconsistent for the rel attribute with the 'nofollow' value. Is this the case. From my research Google, Yahoo and Bing seem to obey this attribute ...