Questions tagged [googlebot]

Googlebot is the bot software that Google uses to crawl over 20 billion pages each day, the data obtained during a crawl is then analyzed and ranked on Google Search.

Filter by
Sorted by
Tagged with
2
votes
0answers
24 views

Googlebot doesn't see links in Angular SPA because routerLink is rendered with a href starting with a hash

In My Angular SPA I have links like: <a routerLink="/en/my-page" ... When I check it in google search console I see that googlebot see the link as: <a href="#/en/my-page" ......
1
vote
0answers
10 views

SEO rawtext backlink [duplicate]

If I add an my http address in a text box of another website and as raw text such that it is not a hyperlink, would this count by googlesbot and for SEO purposes as a backlink? I note a question ...
2
votes
1answer
22 views

What is the longest a page can be in the Google Search cache before it is re-crawled?

The more popular a page is, the more often Googlebot crawls it. For less popular pages, Googlebot can go many months between crawls. If the page were taken down in between crawls, the page could ...
2
votes
2answers
30 views

When I redirect a page with a 301, should I update my Sitemap immediately with the new URL?

Those pages are all indexed and rank well, so I want to be extra careful. This is my current sitemap.xml: /OLD_FOLDER/some-slug-1 /OLD_FOLDER/some-slug-2 /OLD_FOLDER/some-slug-3 I'm setting up 301 ...
0
votes
0answers
14 views

Why google crawler indexes old data of the website? [duplicate]

A month ago I updated the website with a new Title and Description (basically the whole website had been changed) but the Google crawler doesn't index these new data and continues to show the old one. ...
0
votes
0answers
35 views

SEO friendly best practice for putting id and title in different language

I know there are very similar questions like this, but I didn't find the best answerer. I have url like site.com/:id1/:id2/:id3 For having best result in SEO, someone told its better to put titles in ...
4
votes
2answers
59 views

How to keep crawler robots from indexing my page when there is a query string present in the URL?

I have a /products page. In that page, users may use filters to select the products they want to see. Those filters will work using query strings like: /products?brands=X,Y&size=1,2 and so on. I ...
2
votes
0answers
11 views

Does Google use the text property in image object schema markup to determine the relevance of a page to a query?

I'm creating a page that summarizes a variety of statistics on a topic. I believe a dashboard-like graphic like this one would provide a better experience than a textual list of the statistics. ...
1
vote
0answers
37 views

I submitted 500 sitemaps to Google and now it keeps downloading them every 30 minutes nonstop. What is going on?

I used the google.com/ping service to make it download 500 sitemaps for a customer of mine, that was about 30 hours ago. Googlebot reacted immediately, downloaded all 500 sitemaps. Now I have more ...
1
vote
0answers
16 views

How Google decides the image results for the Image tab?

Whenever I search on Google, Google shows the results cards in All tab at SERP and Images at Images tab. So how Google decides which image will be placed at first position or at first row according to ...
5
votes
2answers
623 views

How to avoid robots from indexing pages of my app through alternate URLs?

I have this web app that is accessible through those different URLs. URL_1: www.mycustomdomain.com // THIS IS MY CUSTOM DOMAIN (PUBLIC TO THE USERS) URL_2: www.my-project.firebaseapp.com ...
1
vote
1answer
33 views

Google bot is trying to get crawl a URL of another website on the same machine

I am running multiple websites on a Linode VPS server. I use webmin/virtualmin to manage the websites. Today when I look at Site A's apache access log, I found that there is a record showing Google ...
1
vote
0answers
19 views

Prevent indexing of text loaded by javascript from XHR-ed JSON file

I have a web document, https://example.com/page.html, where a javascript, like <script src="https://example.com/script.js"> loads a <div class="content"></div> ...
1
vote
1answer
29 views

If I hide some scripts from Google bot, will it be considered as cloaking?

I have a website built with Gatsby.js. After building it, I receive a lot of static HTML files, with React app bound to them. All content is in HTML already, JS is needed for some fancy transitions, ...
3
votes
1answer
41 views

Link rel=“nofollow” vs robots.txt disallow for “/search” URLs?

There's a /search path on my web app that I don't want robots to crawl or index. What would be the best approach for this scenario? 1: Block them from accessing by adding a Disallow for /search on ...
0
votes
1answer
22 views

Thousands of visits from Huawei ASN, Singapore. Should I worry about them?

I've got a website with information I do not want to be replicated. I'm browsing my visits, and found thousands of visits per day from AS136907, an ASN of Huawei, Singapore. Always with similar IP ...
0
votes
1answer
30 views

Google SEO crawl budget problem with a website of 2,400,000 URLs: should I keep on serving AMP files?

I manage a website with 200,000+ products, with 2,400,000 URLs: 600,000 URLs for non-AMP visitors in English (per product: 1 page of features, 1 page with photos, 1 page with opinions) 600,000 URLs ...
2
votes
1answer
75 views

Will deleting and re-upload my new site site help Google index it and resolve duplicate URLs that are now fixed?

I uploaded a new site with a domain name https://example.com (actually the domain consists of 3 words concatenated together). If i do a google search for those three words in Google, the site does ...
0
votes
0answers
4 views

Can google index site with js? [duplicate]

I want to start optimizing and SEO a website that seems render in client side. in "view page source" I just see links for CSS and Js. none of the html tags are there. Can Google index the ...
1
vote
2answers
27 views

Does google bot crawl comment author link?

I have one interesting question about comment backlink. As you know, you can leave a comment in wordpress site's post. Then, you can leave a link to Name field or Comment field as below picture. When ...
1
vote
0answers
18 views

Where to put anchor tags in Angular CSR Applications such that routes get indexed? [duplicate]

I built this site with Angular 10: https://fireflysemantics.com/ And when the top level domain is visited it reroutes to this: https://fireflysemantics.com/welcome However There's no link to that from ...
0
votes
0answers
16 views

Where to put anchor tags in Angular CSR Applications such that URLs get indexed?

I built a site with Angular 10. When the home page is visited it reroutes to this /welcome. However There's no link to that from anywhere within the app, so I don't think Google has any way of ...
0
votes
0answers
12 views

Teaching Google to Show a Menu for an Angular App? [duplicate]

If we Google "Panera" google shows Panera Bread as the top ranked result and below the description there's things like "Menu", "Locations", etc. How do we get these ...
2
votes
1answer
50 views

Submit URLs created by our client side rendered Angular site through Google Search Console or will Google automatically find them?

When deploying an Angular App with client side rendering only, do we need to take additional steps for Google to index the app, or is it automatic. I did go to search console and register the site, ...
1
vote
1answer
48 views

htmx and reachability by google bot

Some days ago I found htmx which I really like. It is simple and gives me a way to use my favorite backend solution, and I can avoid a lot of JavaScript. But I am unsure if the google bot will find ...
3
votes
1answer
70 views

List of potential origins of host domains through which google bot crawls?

What is the potential list of origins of the host domains through which google bot crawls? e.g.: Like Google cache uses the following domain to crawl the APIs: http://webcache.googleusercontent.com/ I ...
0
votes
1answer
55 views

Page indexing issues [closed]

I have a problem when indexing in google search engine. There is a page https://frontext.ru/posts/2020-08-27-making-cross-browser-and-responsiveness-better, so a few days later I noticed that it was ...
3
votes
2answers
84 views

How/why did non-www version of page on my site get indexed?

For some reason there are 3 or 4 pages on my site for which Google has indexed the non-www version. I 301 redirect all requests to https and www in .htaccess but it's very odd to me that when I search ...
0
votes
0answers
19 views

Interesting, but weird Google behavior on redirected URLs, if redirect targets are disallowed for crawling

There are two sites: a.example and b.example. URLs a.example/cat were allowed for crawling at a.example/robots.txt. URLs a.example/cat/page were disallowed for crawling at a.example/robots.txt. Then ...
0
votes
0answers
8 views

infinite loader with console error on a page will harm SEO [duplicate]

I have a single page application where some of the category and brand pages are infinite load on the screen (those pages are older now no more available) I want to avoid them. should I use 404 not ...
0
votes
1answer
35 views

What is the best practice for old pages that no longer exist on an Angular site?

We built our eCommerce web site using Angular as a single page application. We have implemented a spinner that runs until the page is responding with the expected content. We have some category pages ...
1
vote
1answer
69 views

I have 2.1 million pages, how to get crawled by googlebot more frequently? [duplicate]

I have a project which aggregates public information on companies, startups, founders etc. for a total of >2 million tracked entities worldwide. Yes, every single page is high quality, manually ...
1
vote
2answers
43 views

Does crawler index duplicate page which has canonical URL?

Suppose I have two pages with the same content: example.com/a example.com/b so I put a canonical tag <link rel="canonical" href="example.com/a"/> at page example.com/b. I ...
0
votes
1answer
39 views

should I set images to no index to avoid crawl budget limit

Most of my posts in search console are showing URL is on Google, but has issues I checked Crawl Page more info section and found out the blog post with many images show other error and the image ...
1
vote
1answer
34 views

Almost Duplicate Description but different images, internal links and meta title and description

I am creating a real estate website and I have thousands of property pages in the similar location. For the property in the same locations, the property descriptions are almost similar like as follows ...
0
votes
0answers
41 views

Google search console consider 404 urls as crawl anomaly category and not 404 not found category

I've got a site which we have mention as 404 not found for a few particular pages so that google can consider and push it to Excluded Not found (404) category, but still, google is considering those ...
1
vote
1answer
21 views

How will copy-pasting articles from my supplier affect my SEO?

My supplier has interesting blog posts on his website, and I'm allowed to copy-paste these articles onto my own website. That's great! But I'm worried about my Google ranking being affected by ...
1
vote
1answer
101 views

Can Googlebot discover a new subdomain through a DNS CNAME chain?

I have a website - www.example.com - which is listed on Google, domain verified in Search Console, sitemaps submitted, canonical urls all in line. This is my only domain that is "out there" ...
2
votes
1answer
25 views

Will Googlebot click a button/link to reveal AJAX content?

my client has to inform it's customers about some new regulations that the Googlebot should NOT crawl. It is not possible to place this information on a separate page and disallow Google to crawl it. ...
0
votes
2answers
21 views

What is the time interval that reputable search engines can hit pages of your website during a crawl?

I'm trying to implement some protection against automated attacks on my website (mostly via scripts, probing for SQL injection, etc.) So I'm checking if two consecutive visits from the same IP fall ...
0
votes
1answer
25 views

Interactive links from AMP to non-AMP without affecting crawl budget

I am maintaining a website with several million pages of user generated content. On each page, there is a calculator implemented allowing people to press a [+] or [-] button to increase or decrease a ...
1
vote
1answer
47 views

How to use Google Indexing API to have Google Bot crawl my pages instantly

I recently came across a few articles that said that using the Google Indexing API will make Google Bot instantly crawl and index all my website's pages. As I have a large number of pages on my site, ...
2
votes
0answers
43 views

GoogleBot constantly crawls non-existant PDFs with 'viagra' or 'cialis' in the name?

My access log is full of requests for non-existent pdfs relating to 'viagra' and 'cialis' or other similar drugs from GoogleBot (user-agent is: Googlebot/2.1 (+http://www.google.com/bot.html) ip range ...
0
votes
2answers
31 views

H1 tag word with two styles

For the purpose of playing with the words I have H1 tag with different styles. Moreover those styles are applied two to one word. For example: <h1>Some<span>thing</span> dif<span&...
1
vote
1answer
45 views

How can a search engine crawl a dynamically generated website?

A dynamically generated website doesn't have most of its content hard coded on the HTML file. Sometimes it waits for a user to login or wait for an input to generate its views. So it would look ...
0
votes
1answer
30 views

Google crawlbot parses &amp; on sitemap.xml wrong

I have a sitemap.xml file with entries of the form: <url> <loc>https://www.example.com/?page=x&amp;item=5</loc> <changefreq>weekly</changefreq> </url> ...
1
vote
0answers
22 views

Will high value of keep-alive help increase Google crawl rate

I have a website with thousands of pages with most of them indexed. The pages receive frequent updates (usually monthly). Google's crawl rate is good, but not enough to capture all the changes before ...
0
votes
1answer
221 views

What does “no data” in search appearance of google webmaster tool/search console mean?

Search appearance section has no data to show. It has been months since some of the pages and posts of my website got indexed to google. But I see no data in search appearance. Does this mean that ...
1
vote
1answer
26 views

How to fix excluded URLs after website was hacked

My website was hacked a while back and google managed to index millions of pages with hacked content. I have requested the removal of pages that Google indexed and still waiting for removal. However,...
1
vote
0answers
11 views

Handling international SEO indexing for “country-specific” and “language-specific” URLs on the same website?

I'm developing an international website and this is my current situation. My home URL is country-specific, i.e: every country has specific content that differs from the others. / USA /au ...

1
2 3 4 5
17