All Questions
3 questions
2
votes
1
answer
306
views
If a web crawler get a 404 error, will it not index that url?
Say I have a page page.example.com/page. In my (python) server, you only get served html if you are logged in, which you can do at page.example.com/login. If you aren't logged in, the server responds ...
- 123
0
votes
1
answer
505
views
Crawl errors for pdf download links in my website
We are building a local business classified website which has about 6000 unique listingsunder huge number of categories and sub-categories.
For each listing there is a details page in which we have ...
- 142
4
votes
2
answers
548
views
Google Webmaster Tools reports fake 404 errors
I have a website where Google Webmaster Tools reports 15,000 links as 404 errors. However, all links return a 200 when I visit them. The problem is, that eventhough I can visit these pages and return ...
- 381