Reputation
Top tag
Next privilege 125 Rep.
Vote down
Badges
3
Newest
 Scholar
Impact
~488 people reached

  • 0 posts edited
  • 0 helpful flags
  • 1 vote cast
Nov
25
comment sitemap that contains millions of links/URLs
I couldn't think of a better way to implement browseable hierarchy just for the crawler. In the case of Amazon, do they provide a full list of products per category?
Nov
25
comment sitemap that contains millions of links/URLs
Your assumptions are absolutely correct. Apparently Amazon has 20 millions of URLs in their sitemap.