On one website I have, looking at the logs, I find lots and lots of "Sosospider" hits. Is this the same for everyone, or just me? Now, I have never once been sent any traffic from anything which looks like it might be anything to do with Sosospider, so can anyone explain what this thing does? Does it cache the pages and send cached pages to users or something?
2 Answers
It looks like it is just a search engine crawler for a website in China. The lack of traffic may have to do with the fact that your site is in English (assumption) and probably isn't what its users are looking for. I would think the crawler wouldn't crawl you since your site is not in Chinese (assumption) but maybe their system isn't that sophisticated or doesn't care.
answered Oct 14, 2010 at 0:08
-
Yes, the site is in English. I also get some hits from Baiduspider but I think I have rarely had an actual hit from them; I have this suspicion that they are caching my pages and serving the cached pages to their users.– deleteCommented Oct 16, 2010 at 13:22
-
Just because you aren't getting any traffic from them doesn't mean they're stealing your content. They're likely just trying to expand their international index. Many Chinese people can read English, just as many Americans read foreign languages. So it makes sense for search engines to index foreign sites. Even if no one is searching for these pages yet, they could in the future. And spidering more pages helps them build better link graphs for ranking results, even if not all pages will be useful results in and of themselves. Commented Jan 28, 2011 at 12:03
Sosospider is a Chinese Search Engine. Whether you think it is worthy of crawling your site depends on your sites contents. My site is an online retail shop which does not ship to China, so we do not allow them to crawl it. However, if you decide to ban them and implement a robots.txt file, keep an eye on your logs because many have said they ignore the robots.txt (regardless of what their website states).
