Skip to main content
6 votes
Accepted

How long does Bingbot typically take to process a sitemap submission?

This post is 4 years old, hope the status in not Pending anymore. But this helped me. After 14 days of pending status, I found this link, to PING bing with your sitemap. It's was a kind of awake call ...
langeleppel's user avatar
4 votes

Why are Bing and SEMRushBot ignoring crawl-delay in my robots.txt file?

Your robots.txt file does not do what (I think) you intend it to do because you do not use blank lines between the sections. The robots.txt standard says (emphasis added): The format logically ...
Stephen Ostermiller's user avatar
4 votes

Is it possible to exclude just OpenAI - ChatGPT from scraping my website?

Yes, it is possible both through a robots.txt declaration and to be restricted at the IP address level. If you want to restrict it via robots.txt, you need the following declarations: User-agent: ...
Maniac's user avatar
  • 324
4 votes

Is it possible to exclude just OpenAI - ChatGPT from scraping my website?

they have a special useragent for their scraper now, it’s GPTBot https://platform.openai.com/docs/gptbot i would also recomment blocking Googlebot, since they declared to use everything they can ...
magnus's user avatar
  • 41
3 votes

Does Bing execute Javascript when crawling and indexing web pages (like Google)?

As of 2018, the Bing Team clarified that bingbot is generally able to render JavaScript. They do recommend detecting the bingbot user agent and prerendering the content on the server side and ...
Scott Willeke's user avatar
3 votes

Bing stopped indexing the site after disallowing all crawlers except Googlebot, Yahoo-slurp, and Msnbot

Msnbot is just one of the user agents that Bing users, and it isn't even the main one. According to https://www.bing.com/webmaster/help/which-crawlers-does-bing-use-8c184ec0 Bingbot - Main bing ...
Stephen Ostermiller's user avatar
3 votes

Inconsistent site: search results on bing and yahoo

Firstly on yahoo if I use site:https://www.example.com/ using https Yahoo declares 121 results. But site:http://www.example.com/ or site:example.com declares 1300 results. Why would this be? ...
DocRoot's user avatar
  • 4,297
3 votes

Why are Bing and SEMRushBot ignoring crawl-delay in my robots.txt file?

Thanks for the question and welcome to this great community of like-minded individuals. You're query is perplexing however it is solvable. I see a couple of things that can be improved/changed in your ...
Nina Dubovich's user avatar
3 votes

Bing refuses to index my site at all

First up, Webmaster Tools is pretty inaccurate. We have a page on there that has been on Bing since the beginning. It claims it was discovered in 2018 (which isn’t true). The error message is ...
Jack Yan's user avatar
3 votes
Accepted

Is it possible to make your website to be searchable with specific keywords?

NO. Webmasters can't definitively do anything to force Googles search to behave a certain way. The best they can do is follow best practices for the moment and provide data optimized in a way that ...
davidgo's user avatar
  • 8,069
2 votes

Google SERP penalty for Multiple Business websites on one server?

absolutely no penalty for "multiple websites, all related to the same business-type, on one server and all of them sharing the same IP address". Different if these multiple website are low quality ...
Massimiliano Rubino's user avatar
2 votes
Accepted

Bing + React: Inconsistent indexing

No. You can't really encourage Bing to index JS properly. It seems like unless you use the isomorphic / universal style of coding javascript apps then you're just out of luck. It's a roll of the ...
Hayk Saakian's user avatar
2 votes

Bing webmaster indicating crawl errors on http to https 301 redirect

...my question is why is it looking at http pages when I have https pages in sitemap... This is the key. The answer is simple. Sitemaps are not the authority by which search engines submits pages to ...
closetnoc's user avatar
  • 32.9k
2 votes

Do all search engines obey nofollow rel attribute on hyperlinks

From my experience, BingBot does follow links despite them having a nofollow attribute. I can see it currently in our logs: thousands of requests per day for pages that are linked using nofollow ...
Effing's user avatar
  • 151
2 votes

Bing returns garbed text for PDFs

Looks like you have not optimised the PDF for search engines as there is no meta information on the document. I have added a screenshot of what search engines will be seeing. This can be updated ...
Jason's user avatar
  • 686
2 votes

Remove site from search engines for specific keyword

I dont know any way to unrank a page for spesific keyword, the only think that would work is removing from your page (H1, Title, Descriptions, p) which are related to the keyword. This is what John ...
John Could's user avatar
2 votes

Website deindexed by Bing.com but they won't tell me why

In my opinion, you should not do anything with it. Go back to it next quarter and see if if hasn't straightened itself out. You seem more informed than 75% of websites. I wouldn't worry about this. ...
Mike Ciffone's user avatar
  • 6,435
2 votes

Bing Caching - 125 KB Limit

I was stumbling above the same SEO warning of Bing's webmaster tools. The description you linked (which is still there at time of writing) indicates that it's about the search crawler internal cache, ...
MichaIng's user avatar
  • 121
2 votes

How to inspect and fix Bing's "Crawl errors"

Uncheck the options aside from the errors, and then click the SEO Reports section to see the errors. They should be listed individually, and a lot of them, (missing alt tags, multiple h1, description ...
keepkalm's user avatar
  • 917
2 votes

NextJS site hosted on Heroku not indexed by Bing after 2 months

Bingbot does not execute JavaScript, so any content on your pages that requires JavaScript to execute on the client isn't going to get indexed in Bing. Bingbot effectively sees a nearly blank home ...
Stephen Ostermiller's user avatar
2 votes

Does Bing penalised sites for too many H1 Tags?

Anecdotally? Only Bing can directly answer your question -- but I believe I can indirectly answer it. I think your experience also answers it, although indirectly. Another way to ask that question is ...
Wayne Smith's user avatar
  • 2,701
2 votes
Accepted

Bing Search Performance Keyword and Page Exports don't add up, should they?

I am able to replicate this on my end, the data doesn't add up. In addition, the data appears to come back in different orders depending on what page I click export from in the report's pagination. ...
Mike Ciffone's user avatar
  • 6,435
2 votes

My website's favicon is not displayed in Bing search results

This is a caching issue. It could take awhile to resolve. Be patient. Note that Bing is insistent on requiring the .ico file extension. While Google will use whatever image you've setup even if it is ...
Justin's user avatar
  • 84
1 vote
Accepted

issues with title and description of my website in Bing results

So Microsoft is very picky. If you check out the link below it will tell you how Bing handles the search results. Typically you can change this via page description in your header. Under the ...
keithpotz's user avatar
1 vote

Negative Canonical Attacks, do they work? how to recover? how to prevent?

Looks like a Negative SEO attack and ive been dealing with one recently for me tracking weekly or monthly backlinks works fine. On your situation i would suggest to disavow the external pages (toxic ...
John Could's user avatar
1 vote

Does Bing execute Javascript when crawling and indexing web pages (like Google)?

Just tested it and Bing found page with all text build purely by JavaScript. Google also finds it of course.
Bobris's user avatar
  • 119
1 vote

Why deny access to website for msnbot/bingbot?

Recent changes in the relationship between Bing -> Edge makes the question interesting. Should we accept the bingbot behaviour? The last couple of weeks we have seen -in northern Europe anyhow -Bing ...
Tom's user avatar
  • 111
1 vote

How to target just one search engine and optimise for that

Here are some ranking criteria that will help you to get better rankings in Bing: Exact match domain or keyword in the URL is the basic rule to get a higher ranking in Bing. Make the keyword ...
Hardeep Singh's user avatar
1 vote

How long does Bingbot typically take to process a sitemap submission?

Dealing with Bing this week, I found out it works, but it's Webmaster tools are just very very delayed. Two days after submitting sitemap, it has status pending, last crawl date is some strange date ...
Mike's user avatar
  • 312
1 vote

How I know my site is indexed on AOL or not

According to this article, as of January 1, 2016, Bing powers AOL search. Therefore, you can submit your site to Bing here: https://www.bing.com/toolbox/submit-site-url You can check to see if your ...
Grant Miller's user avatar
  • 1,183

Only top scored, non community-wiki answers of a minimum length are eligible