I'm looking for a good open source web crawler and i found these:
DataparkSearch, GNU Wget, GRUB, Heritrix, ht://Dig, HTTrack, ICDL, mnoGoSearch, Nutch, Open Search Server, PHP-Crawler, tkWWW Robot, Scrapy, Seeks, YaCy.
But I can not decide which is the best to search for products and prices.
Does anyone have experience with web crawler and could help me?
Only a tip for where i can read something helpful will help so much.
If this question was asked in wrong StackExchange site please please tell me which one is correct.