6
votes
Accepted
How long does Bingbot typically take to process a sitemap submission?
This post is 4 years old, hope the status in not Pending anymore. But this helped me.
After 14 days of pending status, I found this link, to PING bing with your sitemap. It's was a kind of awake call ...
4
votes
Why are Bing and SEMRushBot ignoring crawl-delay in my robots.txt file?
Your robots.txt file does not do what (I think) you intend it to do because you do not use blank lines between the sections. The robots.txt standard says (emphasis added):
The format logically ...
4
votes
Is it possible to exclude just OpenAI - ChatGPT from scraping my website?
Yes, it is possible both through a robots.txt declaration and to be restricted at the IP address level.
If you want to restrict it via robots.txt, you need the following declarations:
User-agent: ...
3
votes
BingPreview user agent
I realize that this is an old post but after researching this I came upon an article explaining the purpose of this user agent.
In this article they explain that this is a user agent string used to ...
3
votes
Inconsistent site: search results on bing and yahoo
Firstly on yahoo if I use site:https://www.example.com/ using https Yahoo declares 121 results. But site:http://www.example.com/ or site:example.com declares 1300 results. Why would this be?
...
3
votes
Bing stopped indexing the site after disallowing all crawlers except Googlebot, Yahoo-slurp, and Msnbot
Msnbot is just one of the user agents that Bing users, and it isn't even the main one. According to https://www.bing.com/webmaster/help/which-crawlers-does-bing-use-8c184ec0
Bingbot - Main bing ...
3
votes
Why are Bing and SEMRushBot ignoring crawl-delay in my robots.txt file?
Thanks for the question and welcome to this great community of like-minded individuals. You're query is perplexing however it is solvable. I see a couple of things that can be improved/changed in your ...
3
votes
Bing refuses to index my site at all
First up, Webmaster Tools is pretty inaccurate. We have a page on there that has been on Bing since the beginning. It claims it was discovered in 2018 (which isn’t true). The error message is ...
3
votes
Is it possible to exclude just OpenAI - ChatGPT from scraping my website?
they have a special useragent for their scraper now, it’s GPTBot
https://platform.openai.com/docs/gptbot
i would also recomment blocking Googlebot, since they declared to use everything they can ...
3
votes
Accepted
Is it possible to make your website to be searchable with specific keywords?
NO. Webmasters can't definitively do anything to force Googles search to behave a certain way. The best they can do is follow best practices for the moment and provide data optimized in a way that ...
2
votes
How to remove deleted webpage returning 200 from indexers?
At this point, your best bet is probably re-purchasing the domain and removing all DNS records. This way search engines (and users) won't be able to access anything; it will be as if nothing exists.
...
2
votes
Google SERP penalty for Multiple Business websites on one server?
absolutely no penalty for "multiple websites, all related to the same business-type, on one server and all of them sharing the same IP address". Different if these multiple website are low quality ...
2
votes
Accepted
Bing + React: Inconsistent indexing
No. You can't really encourage Bing to index JS properly.
It seems like unless you use the isomorphic / universal style of coding javascript apps then you're just out of luck.
It's a roll of the ...
2
votes
Bing webmaster indicating crawl errors on http to https 301 redirect
...my question is why is it looking at http pages when I have https pages in sitemap...
This is the key.
The answer is simple. Sitemaps are not the authority by which search engines submits pages to ...
2
votes
Do all search engines obey nofollow rel attribute on hyperlinks
From my experience, BingBot does follow links despite them having a nofollow attribute. I can see it currently in our logs: thousands of requests per day for pages that are linked using nofollow ...
2
votes
Accepted
Return message from Bing search engine in English with curl php
By default Bing will reply in the native language that it thinks is correct based on the location of the user. This is why you are receiving a response in the language of the region your server is ...
2
votes
Bing returns garbed text for PDFs
Looks like you have not optimised the PDF for search engines as there is no meta information on the document. I have added a screenshot of what search engines will be seeing.
This can be updated ...
2
votes
Remove site from search engines for specific keyword
I dont know any way to unrank a page for spesific keyword, the only think that would work is removing from your page (H1, Title, Descriptions, p) which are related to the keyword.
This is what John ...
2
votes
Website deindexed by Bing.com but they won't tell me why
In my opinion, you should not do anything with it. Go back to it next quarter and see if if hasn't straightened itself out.
You seem more informed than 75% of websites. I wouldn't worry about this.
...
2
votes
Bing Caching - 125 KB Limit
I was stumbling above the same SEO warning of Bing's webmaster tools.
The description you linked (which is still there at time of writing) indicates that it's about the search crawler internal cache, ...
2
votes
How to inspect and fix Bing's "Crawl errors"
Uncheck the options aside from the errors, and then click the SEO Reports section to see the errors. They should be listed individually, and a lot of them, (missing alt tags, multiple h1, description ...
2
votes
NextJS site hosted on Heroku not indexed by Bing after 2 months
Bingbot does not execute JavaScript, so any content on your pages that requires JavaScript to execute on the client isn't going to get indexed in Bing. Bingbot effectively sees a nearly blank home ...
2
votes
Does Bing penalised sites for too many H1 Tags?
Anecdotally?
Only Bing can directly answer your question -- but I believe I can indirectly answer it. I think your experience also answers it, although indirectly.
Another way to ask that question is ...
2
votes
Accepted
Bing Search Performance Keyword and Page Exports don't add up, should they?
I am able to replicate this on my end, the data doesn't add up. In addition, the data appears to come back in different orders depending on what page I click export from in the report's pagination.
...
1
vote
Calculating requests per second within bing
Aside from setting the "Crawl Control" settings within Bing Webmaster tools you could add a crawl-delay: directive to your robots.txt file.
Bing supports whole number values ranging from 1 – 20. ...
1
vote
How do you filter results by site using Bing?
Bing supports the use of the site: filter in the search bar. I just performed a test using Bing and doing a search for a government department in Australia to filter just on departments website for ...
1
vote
How I know my site is indexed on AOL or not
AOL search is powered by bing. When you're on AOL.com and search for something, you can use the parameter site:example.com. It should show you if your site is indexed or not.
You can submit your site ...
1
vote
Bing not indexing URLs From Sitemap
The reason for this is simple.
There are two major schools of thought regarding indexing the web.
One is to index everything and then decide what to keep. The advantage of this is that all metrics ...
1
vote
How long after fixing a crawl error in Bing Webmaster Tools before Bing recrawls and stops showing the error?
Bing, like Google, indexes sites at variable intervals depending on how important it perceives a site to be. So wait times could be as little as one day and as long as several weeks.
I would give it ...
1
vote
Fixed position age verify using cookies without user agent sniffing
Google will likely see the overlay even if you delay it using a multi-second timer. When Googlebot processes JavaScript it seems to be able to skip over (or speed up) timeouts. In the past, I've tried ...
Only top scored, non community-wiki answers of a minimum length are eligible
Related Tags
bing × 171seo × 65
google × 42
bing-webmaster-tools × 33
web-crawlers × 22
yahoo × 22
google-search × 16
search-engines × 16
search-engine-indexing × 14
bingbot × 11
search-results × 10
googlebot × 6
robots.txt × 6
rich-snippets × 5
crawl-errors × 5
url × 4
sitemap × 4
advertising × 4
spam × 4
google-analytics × 3
google-search-console × 3
domains × 3
wordpress × 3
https × 3
links × 3