123 reputation
3
bio website
location
age
visits member for 3 years, 11 months
seen Jul 22 '12 at 2:40

Jan
31
awarded  Scholar
Jan
31
accepted sitemap that contains millions of links/URLs
Nov
25
comment sitemap that contains millions of links/URLs
I couldn't think of a better way to implement browseable hierarchy just for the crawler. In the case of Amazon, do they provide a full list of products per category?
Nov
25
awarded  Supporter
Nov
25
comment sitemap that contains millions of links/URLs
Your assumptions are absolutely correct. Apparently Amazon has 20 millions of URLs in their sitemap.
Nov
24
awarded  Student
Nov
24
asked sitemap that contains millions of links/URLs