My website has been online for a couple years. It's on the first result for Google but it doesn't seem to be indexed by Bing. I tried to fetch it with bing in their webmaster tools and it always returns 'Unexpected Error'. If I delete the robots.txt file the error is "cannot download robots.txt".
-
I checked your robots.txt file and it seems simple enough. The only caveat I see may be the /images/ line. You can temporarily remove this and try again. You can always re-add the line if this does not seem to be the problem.– closetnocNov 21, 2014 at 18:18
Add a comment
|
1 Answer
Bing has a help document that says the following about "Unexpected Error":
What does "Unexpected Error" mean?
If you are seeing frequent "Unexpected Error" results in Fetch as Bingbot, this may indicate that our crawler is not able to connect to your server and not getting a server response at all. Your server may be unintentionally blocking Bingbot based on user agent or IP range. To remedy this, look for server configurations or modules that may be dropping connections for Bingbot or work with your ISP to identify whether the issue is on their side.
Check your firewall. Your website host may also have a firewall that could be blocking the requests.
answered Nov 21, 2014 at 18:15
Stephen Ostermiller on Strike♦Stephen Ostermiller on Strike
97.3k15 gold badges134 silver badges353 bronze badges
-
The problem with the generic help is it is not accurate. For example, I have a site that is hosted by google cloud infrastructure with no restrictions on who can connect. Everyone and everything I can find will successfully connect to my site.– JayJan 21, 2020 at 20:34
-
The second reason there is a problem with this help, is that Bingbot (using the test page in the microsoft control panel area) will successfully get every page on my site except for the front/main/home page of my site. So Bing can connect to my site, it just has a problem with the front page.– JayJan 21, 2020 at 20:36