Last call to make your voice heard! Our 2022 Developer Survey closes in less than a week. Take survey.

Questions tagged [googlebot]

Googlebot is the bot software that Google uses to crawl over 20 billion pages each day, the data obtained during a crawl is then analyzed and ranked on Google Search.

Filter by
Sorted by
Tagged with
-1 votes
0 answers
16 views

Hey! I'd like if you can answer those three simple questions of... (Don't know how to describe that) [closed]

Should I add an a tag before the href? - I have target and href, where put the a tag? - <target="_blank" href="/"> Should I add an a tag before any href in my code? Should I ...
user avatar
2 votes
1 answer
25 views

Can we request google to crawl our website every single day to update its indexes and tags on our domain?

I m thinking of making a blog website and noticed that google search results appear as per Google's indexing upon the crawled website. And this web crawling by google happens like once in 2 weeks to a ...
user avatar
  • 21
3 votes
1 answer
37 views

How can pages behind a search input field be accessible to search engine bots?

Maybe the situation I am facing is weird, but I have an application where product pages are found via an search input field and there isn't any kind of products list with their respective links ...
user avatar
  • 33
0 votes
0 answers
6 views

URL is not on Google [duplicate]

I have created a website where I create blog posts, however, the website has pages that are discovered but not indexed. Does that mean that the webserver is not good enough to handle the load when ...
user avatar
0 votes
0 answers
33 views

Getting Visitors' IP Addresses and Showing Related Content

I want to list certain things for different cities. In other words, when someone from New York visits my website, I want to list "Best Language Courses in New York". I can do that by using ...
user avatar
0 votes
0 answers
9 views

How much time Google takes to update new title of website on search engine result page? [duplicate]

I'm learning SEO and performing some basic stuff like creating a catchy title for the homepage. I changed the title of my homepage from (let's assume) 'I am an old title' to 'I am new.' However, when ...
user avatar
  • 1
2 votes
2 answers
48 views

How to redirect without Google following it?

I originally asked this on StackOverflow, but got told to move it here, so here I am. I'm running a host and want to redirect visitors to a page on my main site if the client hasn't set up any content ...
user avatar
  • 23
1 vote
0 answers
17 views

Does Googlebot see duplicated content when content is shown in a loop with JavaScript?

One of the projects I have recently developed uses SwiperJS slider with loop option set to true. It generates duplicated slides which makes sense. Since each slide represents an article the SEO guy ...
user avatar
2 votes
0 answers
103 views

SEO | VueJS | Why do I see "We're sorry but desk-app doesn't work properly without JavaScript enabled. Please enable it to continue." in web browser?

I set up a single page application using Vue.js and Firebase. It works fine but when I search for the website in Google Chrome, this is what pops up : I expect to see a proper description, instead of ...
user avatar
  • 21
0 votes
1 answer
28 views

Ratio of real visitors to search engine views

I know it is highly website-dependent, but is there any statistics/study/report on the ratio of real visitors websites get and the visits made by the search engine crawler? Services like Alexa reports ...
user avatar
  • 442
1 vote
1 answer
20 views

In a client-side-rendered app, some pages are successfully crawled by Googlebot while others fail to fully render. How can I get all pages to render?

The /home and /home/search pages render fine, but /home/product/ID_XYZ pages appear as just a loading icon when doing a "Live Test" in Google Search Console. There has been some flux ...
user avatar
  • 141
0 votes
0 answers
13 views

google crawling the link which is dynamically changed

I have a question about how Google crawls. I have a pagination link on the site. Like this. www.domeinnaam.com/category?page=2 I suppose that google by crawling is going to click on this link. So far ...
user avatar
1 vote
0 answers
16 views

Can I use a single page as the canonical of many image galleries that Google has decided not to index to improve rankings?

I am a hobbyist who is building a website to showcase a wedding photographer's portfolio. On the portfolio site, e.g www.example.com/portfolio, all the individual wedding galleries are listed and ...
user avatar
  • 11
0 votes
0 answers
11 views

HOW TO GET MUTIPLE PAGES INDEXED [duplicate]

I have a website with approximately 700k web pages but only has 2k indexed pages. I have tried increasing the crawl limit to 2 requests per second because that is what google search console allows for ...
user avatar
  • 1
1 vote
0 answers
33 views

SEO test tool doesn't see my H1 generated by JavaScript, will Google?

On what basis is SEO checked? When I examine my page, an H1 tag is present. However, when I look at the page source, an H1 heading is not present because it is generated via JavaScript (CMS in the ...
user avatar
  • 11
2 votes
1 answer
66 views

Are pre-rendering or server-side rendering needed for SEO of a React application?

I am a java developer who is upskilling myself and learning React, so that I can develop frontend application completely using React. But I read couple of articles which states that it could cause SEO ...
user avatar
4 votes
1 answer
67 views

Is there a way to allow ad bots to crawl a website that has "noindex,nofollow" on it?

Due to duplicated content, there is a "noindex,nofollow" on the website, but in order to run ppc ads, there needs to be a way for the ads bots to crawl the website Can this be done without ...
user avatar
4 votes
3 answers
51 views

Does Googlebot call every link separately (with a new request)?

Does Googlebot call every link separately (with a new request) or does it follow all links by JavaScript (for example by click event)? I have a vue-js website which contains all the links and of ...
user avatar
2 votes
2 answers
63 views

Cannot upload sitemap from other domain in Google Search Console

In the screenshot of Google Search Console below, I am looking at my property at example.com/my-site. The black scribble is my site at example.com/my-site, and the yellow is me attempting to link my ...
user avatar
  • 141
2 votes
1 answer
32 views

Is it possible to disable indexing through JS?

Is it possible to disable Google indexing of some web page if this is placed to head tag <meta name="googlebot" content="noindex" /> using only JS? Or is it all a hoax and ...
user avatar
0 votes
0 answers
9 views

Sitemap in robots.txt is not appearing in Google Search Console, no URLs detected in sitemap [duplicate]

At the bottom of robots.txt I have Sitemap: https://domain2.com/sitemap.xml. In the Google Search Console for my property at domain1 I see a false HTML sitemap with errors, no trace of my actual ...
user avatar
  • 141
1 vote
2 answers
26 views

Does PageSpeed Insights performance score somehow affect your website SEO? How much?

I often see that popular websites have red-orange performance scores in PageSpeed Insights. So I'm wondering if that somehow affect SEO or not?
user avatar
1 vote
0 answers
22 views

Disallow crawling of all links to a domain, except only specific URLs with parameters

I have a domain like example.com under which everything is blocked and links to an SPA. Like example.com, and we've disabled everything like so: User-agent: * Disallow: / Now this SPA uses parameters ...
user avatar
0 votes
0 answers
20 views

Google crawler doesn't index multilingual pages

I need a help with sitemaps and multilingualism. I have a web store the operates on three markets (German, Austrian and Slovenian) and the languages available are English, German and Slovenian. I ...
user avatar
1 vote
0 answers
29 views

How can I let Google crawl my site for Programmable Search Engine (custom site search), but not have it show in public search results?

The answer for my question might be inherent in the question itself- that if I want Google to crawl my site for a custom site search, such as through Programmable Search Engine, that it is those same ...
user avatar
4 votes
1 answer
51 views

Would text references to unsafe (http) urls negatively affect SEO?

I've read from numerous sources that unsafe hyperlink urls that use the http scheme may cause google to flag your website as having 'unsafe content'. This is why my website does not allow its users to ...
user avatar
  • 291
2 votes
2 answers
65 views

What happens when you submit a sitemap to google for a website that is not up yet?

I'm building an angular web application that is able to periodically autogenerate sitemaps. Once it is done autogenerating all the sitemaps it automatically informs google of the location of the ...
user avatar
  • 291
0 votes
0 answers
32 views

How to force google to remove the outdated or removed pages from indexing

Although it appears a duplicate question, it is slightly different, application developed using html/javascript contains "html" extension for all the pages and when it is migrated using ...
user avatar
4 votes
1 answer
3k views

What is .well-known/traffic-advice directory?

I'm getting lots of requests from Googlebot on the .well-known/traffic-advice directory on my server. What is it and what are they looking for? Should I add something to this directory, and if yes ...
user avatar
  • 173
3 votes
1 answer
56 views

Does recaptcha version 3 detect and block Googlebot as a bot?

I am trying to protect my back-end resources using re-captcha from misuse from bots however am not sure of its implication on SEO. There is little information clarifying about this issue. From re-...
user avatar
  • 175
1 vote
1 answer
31 views

Googlebot returning 200 and 300 for the same URLs

Is there a reason why a URL on our website would sometimes return 200 and sometimes return a 301? This appears to happen on a number of our URLs even something basic like example.com/robots.txt When I ...
user avatar
  • 273
2 votes
2 answers
290 views

Does submitting sitemap.xml once change the way the site's new pages will be crawled/discovered in the future?

My static site doesn't generate sitemap.xml automatically. Hundreds of pages have been modified recently (meta description, etc.). I would like Google to recrawl the whole website. Submitting the URLs ...
user avatar
  • 532
0 votes
0 answers
5 views

What Is The Reasons That GSC Shows "Page Resources Couldn't Be Loaded" In Live Test? [duplicate]

I Edit My robots.txt File And Put This Code In It: User-agent: * Allow: / Sitemap: https://example.com/sitemap.xml But The Issue Is Not Fixed. Sometimes My CSS File, Sometimes JS File And Sometimes ...
user avatar
2 votes
1 answer
145 views

Does ARIA help SEO? Have you experienced any benefits for organic search?

I've seen mention of ARIA benefitting SEO, but the author cites no evidence — none of the articles cited mention this. I have also searched and haven't found any reliable resources to suggest that ...
user avatar
  • 1,402
3 votes
1 answer
175 views

Is there a way to check how Googlebot would render non-public pages?

I have a page which uses react, and from what I understood docs, Google bot, before indexing, is executing JS (see here: JavaScript SEO Basics). First thing which I want to ask, is if my thinking is ...
user avatar
  • 131
3 votes
2 answers
46 views

Linking relevant ecommerce links with nofollow

I have a price comparison website which lists different products. On the product pages it lists 10-20 webshops per product where the item can be purchased. At the moment I am linking these sites with ...
user avatar
  • 337
3 votes
1 answer
47 views

Does google index content that's inserted using setTimeout after the page is loaded?

Let's say we have this code <body> content-1 </body> <script> setTimeout(changeBodyContentTo("content-2"), 5000); </script> In this case will google crawl and ...
user avatar
  • 33
0 votes
1 answer
26 views

<title> tag and favicon aren't correctly showed by Google Search [duplicate]

I'll go straight to the point. For a few days I've been working on the SEO on my website and I noticed that the favicon still isn't showed by Google Search and that the <title> tag is showed ...
user avatar
4 votes
1 answer
53 views

How to specify paywall contents for multiple pages

I am reading this guide to apply paywall contents to my website so that Google knows I am not cloaking. Unless I have misunderstood something, the example in that guide only deals with 1 particular ...
user avatar
  • 309
4 votes
1 answer
214 views

Does google follow links discovered in JSON-LD?

Lets say I have a site with a page of 20 products which are fed through an API. The products are loaded using infinity scroll so the initial page render has 5 products tops. Upon scrolling it adds 5 ...
user avatar
  • 175
1 vote
1 answer
30 views

Why does not googlebot crawl my website? [duplicate]

I create some websites and ping them to google using pingomatic. Some websites are soon indexed by google, but others are not indexed at all(site: does not return any result). Note that this is not ...
user avatar
  • 153
7 votes
4 answers
3k views

Why is preventing Googlebot from crawling some pages on my site good for SEO?

Google says that it is effective to prevent Google Crawling Bot from crawling some useless pages in my site. https://developers.google.com/search/docs/advanced/guidelines/how-search-works https://...
user avatar
4 votes
1 answer
73 views

Weird bot requests in logs

I restored a website on a new server after it was down (mistakenly deleted) for a while, and it immediately got bombarded with requests to non-existing pages like /tissue-salts-for-weight-loss.html ...
user avatar
3 votes
0 answers
60 views

Why are pages of this sub-domain indexed by google? [closed]

If you site:news.cnblogs.com in google's search box, you will see a lot of web pages are indexed by google. But if you click on any of the search results, you will be redirected to account.cnblogs.com ...
user avatar
  • 153
4 votes
1 answer
131 views

How to allow Googlebot to crawl paywall content?

My website has pages for around 1000 companies. Without an account, I only allow users to view 5 company pages a month (I track them by cookie). And Googlebot can crawl my site freely since it does ...
user avatar
  • 309
1 vote
1 answer
43 views

If noindex pages don't have H1, meta tags, or other desirable elements for SEO, will it hurt the SEO of indexed pages?

I have some pages with noindex. These pages have some SEO errors. SEO errors include: not using H1 Not using meta tags Not adding alt="" tags to images But there no such SEO problems on my ...
user avatar
  • 654
1 vote
1 answer
29 views

Can Googlebot crawl content from JS fetch?

In my website for many pages, on page load I would call fetch in JS to get the data from the server and then render it. I wonder whether Googlebot can crawl content from fetch? I have tried to search ...
user avatar
  • 309
1 vote
0 answers
95 views

Can Google bot index pages built with websocket if I implement dynamic rendering?

This is the first time I build a website so I did not know anything about SEO when I started. I built many pages of my website to be rendered by React + Redux with the data received from websocket (...
user avatar
  • 309
3 votes
2 answers
210 views

Does Googlebot simulate page scroll to load additional content?

The content of the home page of the project I'm building is divided into 6 parts where the user scrolls and as soon as the first part "section" reaches the end another part is added via AJAX,...
user avatar
2 votes
1 answer
152 views

Google Search Console and sitemap-index file gzip

I googled for a long time and tried to find something at https://www.sitemaps.org/protocol.html#index about gzip compression for sitemap INDEX files, but I couldn't find anything. I have a very large ...
user avatar

1
2 3 4 5
18