6
votes
1answer
317 views

robots.txt blocking specific files also block unnecessary URLs

I am using Magento for one of my site. In Magento there is a file mage (no extension file name is mage only) to block this file I write robots.txt as # Files User-agent: * Disallow: /mage But this ...
0
votes
1answer
78 views

Google Webmaster Tool: “Severe health issues are found on your site” [closed]

I just saw the "Severe health issues are found on your site. Is robots.txt blocking important pages?" message on Google Webmaster Tools. This is what I have in robots.txt: User-agent: * Allow: ...
1
vote
1answer
129 views

Ahrefs is checking my site for backlinks all the time

I have a Magento site and I realize that the ahrefs.com crawler visits my site all the time everyday. I visit ahrefs.com and I check my backlinks (they are good). Is it good or bad, something like ...
4
votes
2answers
257 views

Google Webmaster Tools reports fake 404 errors

I have a website where Google Webmaster Tools reports 15,000 links as 404 errors. However, all links return a 200 when I visit them. The problem is, that eventhough I can visit these pages and return ...