Questions tagged [googlebot]

Googlebot is the bot software that Google uses to crawl over 20 billion pages each day, the data obtained during a crawl is then analyzed and ranked on Google Search.

Filter by
Sorted by
Tagged with
0 votes
0 answers
9 views

Are .JSON and сhunk .js in Next.js files important for Googlebot to render page correctly?

I have a problem with Googlebot indexing my website. He scans a lot of JSON and chunks .js files instead of scanning. HTML files. These unnecessary files reduce the crawl budget of my website. So ...
user avatar
0 votes
0 answers
5 views

Error: Missing region-independant link for that language? [duplicate]

Hi this continuation for this question, I generated a sitemap like this: When I try to validate this sitemap, I'm getting the below warning: Missing region-independant link for that language (en) If ...
user avatar
  • 131
3 votes
1 answer
30 views

Which url(s) need to be included in sitemap.xml if I have different urls for different regions?

I have a .com website. I have some lamda function on AWS which redirect URL to specific country URL Ex: for India: www.example.com/in for US: www.example.com.us I have up to 12 geographical regions ...
user avatar
  • 131
2 votes
1 answer
13 views

Google Mobile Friendly Tester Issue - Preview Shown Is Very Small

The mobile friendliness tester is saying that my text is too small, and that clickable elements are too close together. I ensured that ALL text on the page was above 18 font to test, and Google still ...
user avatar
  • 121
1 vote
1 answer
28 views

Does Googlebot index closed HTML details element?

I have the following HTML details element that is closed by default. <details> <summary>Details</summary> <p> Something small enough to escape casual notice. </p&...
user avatar
  • 111
1 vote
1 answer
21 views

All-In-One-SEO plugin sitemap is blank

recently I was working on a project and needed to upload the sitemap to Google search console, I am using All-in-One-SEO plugin for generating sitemap and SEO of this site. However when I go to the ...
user avatar
2 votes
1 answer
35 views

Can Google index page that is login-required and redirects to Login page?

I have one link that is login-required http://example.com/secret If user access the page without authentication it will be redirected to http://example.com/login page. My question is can Google bot ...
user avatar
-1 votes
1 answer
499 views

Does googlebot crawl the variable name of a php [closed]

Does Googlebot crawl the dynamic variable name aside from its value? For example in my PHP I use the variable _company_name. In the front-end is is _company_name saved in bot crawl also?
user avatar
1 vote
0 answers
32 views

Googlebot tries to access links from another unavailable website

I'm getting a bit paranoid because there's been a small breach on my web server recently and that linked to this I found multiple logs like this from nginx: access.log-20220602:66.249.70.69 - - [01/...
user avatar
1 vote
1 answer
29 views

Skip campaigns from indexing

I have a webpage with campaigns that are only valid for a specific period of time. My problem is that when a campaign is active, the homepage redirects to the campaign page and therefore the Google ...
user avatar
2 votes
1 answer
29 views

Can we request google to crawl our website every single day to update its indexes and tags on our domain?

I m thinking of making a blog website and noticed that google search results appear as per Google's indexing upon the crawled website. And this web crawling by google happens like once in 2 weeks to a ...
user avatar
  • 21
3 votes
1 answer
40 views

How can pages behind a search input field be accessible to search engine bots?

Maybe the situation I am facing is weird, but I have an application where product pages are found via an search input field and there isn't any kind of products list with their respective links ...
user avatar
  • 33
0 votes
0 answers
7 views

URL is not on Google [duplicate]

I have created a website where I create blog posts, however, the website has pages that are discovered but not indexed. Does that mean that the webserver is not good enough to handle the load when ...
user avatar
0 votes
0 answers
35 views

Getting Visitors' IP Addresses and Showing Related Content

I want to list certain things for different cities. In other words, when someone from New York visits my website, I want to list "Best Language Courses in New York". I can do that by using ...
user avatar
0 votes
0 answers
9 views

How much time Google takes to update new title of website on search engine result page? [duplicate]

I'm learning SEO and performing some basic stuff like creating a catchy title for the homepage. I changed the title of my homepage from (let's assume) 'I am an old title' to 'I am new.' However, when ...
user avatar
  • 1
2 votes
2 answers
50 views

How to redirect without Google following it?

I originally asked this on StackOverflow, but got told to move it here, so here I am. I'm running a host and want to redirect visitors to a page on my main site if the client hasn't set up any content ...
user avatar
  • 23
1 vote
0 answers
18 views

Does Googlebot see duplicated content when content is shown in a loop with JavaScript?

One of the projects I have recently developed uses SwiperJS slider with loop option set to true. It generates duplicated slides which makes sense. Since each slide represents an article the SEO guy ...
user avatar
2 votes
0 answers
360 views

SEO | VueJS | Why do I see "We're sorry but desk-app doesn't work properly without JavaScript enabled. Please enable it to continue." in web browser?

I set up a single page application using Vue.js and Firebase. It works fine but when I search for the website in Google Chrome, this is what pops up : I expect to see a proper description, instead of ...
user avatar
  • 21
0 votes
1 answer
29 views

Ratio of real visitors to search engine views

I know it is highly website-dependent, but is there any statistics/study/report on the ratio of real visitors websites get and the visits made by the search engine crawler? Services like Alexa reports ...
user avatar
  • 442
1 vote
2 answers
40 views

In a client-side-rendered app, some pages are successfully crawled by Googlebot while others fail to fully render. How can I get all pages to render?

The /home and /home/search pages render fine, but /home/product/ID_XYZ pages appear as just a loading icon when doing a "Live Test" in Google Search Console. There has been some flux ...
user avatar
  • 151
0 votes
0 answers
14 views

google crawling the link which is dynamically changed

I have a question about how Google crawls. I have a pagination link on the site. Like this. www.domeinnaam.com/category?page=2 I suppose that google by crawling is going to click on this link. So far ...
user avatar
1 vote
0 answers
17 views

Can I use a single page as the canonical of many image galleries that Google has decided not to index to improve rankings?

I am a hobbyist who is building a website to showcase a wedding photographer's portfolio. On the portfolio site, e.g www.example.com/portfolio, all the individual wedding galleries are listed and ...
user avatar
  • 11
0 votes
0 answers
11 views

HOW TO GET MUTIPLE PAGES INDEXED [duplicate]

I have a website with approximately 700k web pages but only has 2k indexed pages. I have tried increasing the crawl limit to 2 requests per second because that is what google search console allows for ...
user avatar
  • 1
1 vote
0 answers
37 views

SEO test tool doesn't see my H1 generated by JavaScript, will Google?

On what basis is SEO checked? When I examine my page, an H1 tag is present. However, when I look at the page source, an H1 heading is not present because it is generated via JavaScript (CMS in the ...
user avatar
  • 11
2 votes
1 answer
81 views

Are pre-rendering or server-side rendering needed for SEO of a React application?

I am a java developer who is upskilling myself and learning React, so that I can develop frontend application completely using React. But I read couple of articles which states that it could cause SEO ...
user avatar
4 votes
1 answer
74 views

Is there a way to allow ad bots to crawl a website that has "noindex,nofollow" on it?

Due to duplicated content, there is a "noindex,nofollow" on the website, but in order to run ppc ads, there needs to be a way for the ads bots to crawl the website Can this be done without ...
user avatar
4 votes
3 answers
57 views

Does Googlebot call every link separately (with a new request)?

Does Googlebot call every link separately (with a new request) or does it follow all links by JavaScript (for example by click event)? I have a vue-js website which contains all the links and of ...
user avatar
2 votes
2 answers
67 views

Cannot upload sitemap from other domain in Google Search Console

In the screenshot of Google Search Console below, I am looking at my property at example.com/my-site. The black scribble is my site at example.com/my-site, and the yellow is me attempting to link my ...
user avatar
  • 151
2 votes
1 answer
43 views

Is it possible to disable indexing through JS?

Is it possible to disable Google indexing of some web page if this is placed to head tag <meta name="googlebot" content="noindex" /> using only JS? Or is it all a hoax and ...
user avatar
0 votes
0 answers
10 views

Sitemap in robots.txt is not appearing in Google Search Console, no URLs detected in sitemap [duplicate]

At the bottom of robots.txt I have Sitemap: https://domain2.com/sitemap.xml. In the Google Search Console for my property at domain1 I see a false HTML sitemap with errors, no trace of my actual ...
user avatar
  • 151
1 vote
2 answers
26 views

Does PageSpeed Insights performance score somehow affect your website SEO? How much?

I often see that popular websites have red-orange performance scores in PageSpeed Insights. So I'm wondering if that somehow affect SEO or not?
user avatar
1 vote
0 answers
26 views

Disallow crawling of all links to a domain, except only specific URLs with parameters

I have a domain like example.com under which everything is blocked and links to an SPA. Like example.com, and we've disabled everything like so: User-agent: * Disallow: / Now this SPA uses parameters ...
user avatar
0 votes
0 answers
21 views

Google crawler doesn't index multilingual pages

I need a help with sitemaps and multilingualism. I have a web store the operates on three markets (German, Austrian and Slovenian) and the languages available are English, German and Slovenian. I ...
user avatar
1 vote
0 answers
31 views

How can I let Google crawl my site for Programmable Search Engine (custom site search), but not have it show in public search results?

The answer for my question might be inherent in the question itself- that if I want Google to crawl my site for a custom site search, such as through Programmable Search Engine, that it is those same ...
user avatar
4 votes
1 answer
52 views

Would text references to unsafe (http) urls negatively affect SEO?

I've read from numerous sources that unsafe hyperlink urls that use the http scheme may cause google to flag your website as having 'unsafe content'. This is why my website does not allow its users to ...
user avatar
  • 291
2 votes
2 answers
67 views

What happens when you submit a sitemap to google for a website that is not up yet?

I'm building an angular web application that is able to periodically autogenerate sitemaps. Once it is done autogenerating all the sitemaps it automatically informs google of the location of the ...
user avatar
  • 291
0 votes
0 answers
35 views

How to force google to remove the outdated or removed pages from indexing

Although it appears a duplicate question, it is slightly different, application developed using html/javascript contains "html" extension for all the pages and when it is migrated using ...
user avatar
5 votes
2 answers
4k views

What is .well-known/traffic-advice directory?

I'm getting lots of requests from Googlebot on the .well-known/traffic-advice directory on my server. What is it and what are they looking for? Should I add something to this directory, and if yes ...
user avatar
  • 183
3 votes
1 answer
61 views

Does recaptcha version 3 detect and block Googlebot as a bot?

I am trying to protect my back-end resources using re-captcha from misuse from bots however am not sure of its implication on SEO. There is little information clarifying about this issue. From re-...
user avatar
  • 175
1 vote
1 answer
37 views

Googlebot returning 200 and 300 for the same URLs

Is there a reason why a URL on our website would sometimes return 200 and sometimes return a 301? This appears to happen on a number of our URLs even something basic like example.com/robots.txt When I ...
user avatar
  • 273
2 votes
2 answers
318 views

Does submitting sitemap.xml once change the way the site's new pages will be crawled/discovered in the future?

My static site doesn't generate sitemap.xml automatically. Hundreds of pages have been modified recently (meta description, etc.). I would like Google to recrawl the whole website. Submitting the URLs ...
user avatar
  • 532
0 votes
0 answers
5 views

What Is The Reasons That GSC Shows "Page Resources Couldn't Be Loaded" In Live Test? [duplicate]

I Edit My robots.txt File And Put This Code In It: User-agent: * Allow: / Sitemap: https://example.com/sitemap.xml But The Issue Is Not Fixed. Sometimes My CSS File, Sometimes JS File And Sometimes ...
user avatar
2 votes
1 answer
209 views

Does ARIA help SEO? Have you experienced any benefits for organic search?

I've seen mention of ARIA benefitting SEO, but the author cites no evidence — none of the articles cited mention this. I have also searched and haven't found any reliable resources to suggest that ...
user avatar
  • 1,402
3 votes
1 answer
206 views

Is there a way to check how Googlebot would render non-public pages?

I have a page which uses react, and from what I understood docs, Google bot, before indexing, is executing JS (see here: JavaScript SEO Basics). First thing which I want to ask, is if my thinking is ...
user avatar
  • 131
3 votes
2 answers
46 views

Linking relevant ecommerce links with nofollow

I have a price comparison website which lists different products. On the product pages it lists 10-20 webshops per product where the item can be purchased. At the moment I am linking these sites with ...
user avatar
  • 337
3 votes
1 answer
51 views

Does google index content that's inserted using setTimeout after the page is loaded?

Let's say we have this code <body> content-1 </body> <script> setTimeout(changeBodyContentTo("content-2"), 5000); </script> In this case will google crawl and ...
user avatar
  • 33
0 votes
1 answer
27 views

<title> tag and favicon aren't correctly showed by Google Search [duplicate]

I'll go straight to the point. For a few days I've been working on the SEO on my website and I noticed that the favicon still isn't showed by Google Search and that the <title> tag is showed ...
user avatar
4 votes
1 answer
53 views

How to specify paywall contents for multiple pages

I am reading this guide to apply paywall contents to my website so that Google knows I am not cloaking. Unless I have misunderstood something, the example in that guide only deals with 1 particular ...
user avatar
  • 309
4 votes
1 answer
235 views

Does google follow links discovered in JSON-LD?

Lets say I have a site with a page of 20 products which are fed through an API. The products are loaded using infinity scroll so the initial page render has 5 products tops. Upon scrolling it adds 5 ...
user avatar
  • 175
1 vote
1 answer
32 views

Why does not googlebot crawl my website? [duplicate]

I create some websites and ping them to google using pingomatic. Some websites are soon indexed by google, but others are not indexed at all(site: does not return any result). Note that this is not ...
user avatar
  • 153

1
2 3 4 5
18