Stack Exchange Network

Stack Exchange network consists of 175 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [googlebot]

Googlebot is the bot software that Google uses to crawl over 20 billion pages each day, the data obtained during a crawl is then analyzed and ranked on Google Search.

3
votes
2answers
30 views

How to force a view of meta data as Google or Bing sees it?

I'm having an extremely peculiar problem with a site (WordPress) that just suddenly started exhibiting an issue where Japanese is showing up for all the metadata in Google and Bing - only: Google: ...
0
votes
0answers
19 views

Google is marking as duplicate my unique URLs powered by Angular Universal

I have a problem in the search console, and google is marking my unique urls as duplicate The application is build with Angular Universal and is rendered in the server Note: This site is only ...
0
votes
0answers
24 views

SEO Ajax loaded pages - noindex X-Robots-tag

I have on my homepage (page A) html content (ajax) loaded on click of a button (load more) from multiple pages (page B). Googlebot does see content and does follow links loaded from the ajax calls on ...
0
votes
0answers
18 views

Googlebot crawl anomaly due to blocked resources by Zopim

I am using Zopim live chat to enable chat support in my website, but Googlebot is showing error due to some unreachable resources from zopim service. I have attached a screen capture of the error. ...
4
votes
1answer
72 views

Cookie consent overlay - Does it affect SEO?

I have a german website with Adsense integrated. To meet Google's compliance requirements, I ask users to accept cookies for personal advertising. To achieve this, I created an overlay which stays in ...
2
votes
1answer
42 views

Is blocking googlebot from accessing portions of the site with robots.txt penalized by Google?

I was reading this question. Let's say i have a div with something useful for users but i don't want it to be considered by Google. If i put this on an iframe and block the googlebot with robots.txt ...
1
vote
1answer
36 views

Is crawl budget and link juice being wasted to URLs with hashes?

On my site I have internal links to URLs with hashes in them, like this one: https://example.com/page.html#music https://example.com/page.html#music has canonical tag with URL pointing to https://...
0
votes
0answers
24 views

Googlebot can't index my hbs based SPA website

As part of a school project, I have made a single-page application website using server-side rendered handlebars. However, when inspecting the site using Google Search Console, all Google can find is ...
0
votes
0answers
21 views

Referencement issue with irrelevant suffix

I have a problem concerning the "coverage" on Google Search Console. The website on the company I'm working for have weird errors on this page. Google is telling that it had issues referencing pages ...
3
votes
1answer
35 views

Would using mouse movement to change how the page is rendered be considered “cloaking” to Google?

My site can enhance itself to client side rendered (CSR) (ish) if a prediction can be made by mouse movement analysis. It can gauge a visitor's familiarity with the site, confusion, etc. The content ...
1
vote
1answer
22 views

Will keywords written into a page using JavaScript replacing placeholders in the page source affect SEO?

Website pages use SEO content (words) that are dynamically generated. Like the store name (on a multi store site). However the dynamically generated words only show when you inspect element, as below....
1
vote
1answer
56 views

Google's Inspect URL shows incorrect source code for my page's “Google Index”, but correct for “Live Test”

My web page is incorrectly indexed and is showing a different header than the one that is actually updated in my website. When I used the URL Inspection Tool in Google Search Console, The index shows ...
1
vote
1answer
47 views

Crawled page's screenshot is blank in the Google Search Console

I set up a SPA page through Angular, and achieved the effect of Server Side Render through Rendertron, In the facebook debug tool, I could get my dynamically generated Meta data, but the preview of ...
2
votes
1answer
42 views

Redirecting a user but not Googlebot to solve problem with iFrames in content

TLDR To work around a problem with a third party documentation writing software I need to redirect a page to another page but only if a user in browser is looking at it. I want the google bot to see ...
2
votes
0answers
395 views

New Google Search Console does not load any resources such as CSS and JS while it worked fine in the old Search Console tool

When using URL inspection and Live Test in the new Google Search Console the Googlebot does not load most resources (Javascript, CSS and Images). This results in the Search Console telling me my pages ...
1
vote
0answers
22 views

When using an image server on a separate domain, should we link directly to the image URL and restrict crawling, or redirect through our domain?

I have a large number of images stored on a separate server, these URLs currently go through a 302 redirect. I think these additional pages are a total waste of time for Google to crawl. At the ...
0
votes
0answers
12 views

Only one page got indexed in the submitted sitemap [duplicate]

Google indexed only one URL in the sitemap I submitted. Please let me know if there's anything wrong with my sitemap. Any help appreciated. Example sitemap.xml <!-- language: xml --> <?xml ...
2
votes
2answers
41 views

Do non-existent 'category' pages in URLs present a problem for search engines?

Say for example if I have a news site and my URL structure is: example.com/news/random-article If there is no page at example.com/news/, in other words it results in a 404 error, is that problematic ...
0
votes
0answers
20 views

My website's name doesn't appear on google [duplicate]

My website has been online for 10 days. (content complete) I've allowed Google boots. I also sent a sitemap to the search console. (I use yoast seo by the way) In this 10 days pages were added to ...
3
votes
1answer
48 views

Google not showing cache option on the following url in SERP results?

Google is not showing option to check cache on the following url that I have mentioned below in SERP results.
2
votes
1answer
27 views

Can I use robots.txt to allow Google to access some folders, but disallow everything else?

I want to allow Google bots to crawl only my index.php and the content of some folders. Would that work? User-agent: * Allow: /index*.php Allow: /folder1/ Allow: /folder2/ Allow: /folder3/ Allow: /...
1
vote
1answer
69 views

Why are some google snippet results bold? [closed]

Recently, when I search for keyword "best mlm software" I got a featured snippet having "List of Multi Level Marketing Software". I noted that two of list's item were BOLD text. Didn't know why??? Can ...
1
vote
1answer
37 views

Does Google Crawl Rate have an impact on Google's Image Bot

I have a relatively large website. Each page has several images. Google indexes my website very slowly, but it actively crawls my pictures. Does changing the Google Crawl Rate have an impact on Google'...
2
votes
1answer
63 views

Is it good for SEO to change an article's slug when changing its title?

I have the following URL view: example.com/article/{slug}-{id} If I change the title of the article, then the slug will also change so that there is no disagreement. Please note that the article ...
2
votes
2answers
36 views

How many levels of sub-pages do Google bots crawl to when fetching and indexing linked pages in Google Search Console?

Let's say I submit a sitemap page that contains all the links I want Google to crawl, but instead of submitting a sitemap, I fetch the page and linked pages and let Google index them. How many levels ...
2
votes
0answers
246 views

Discus comments cause “clickable elements too close together” in Google Search Console mobile usabality report

I just got an email from Google that they found some problems on my site: clickable elements too close together Content wider than screen I fetched problematic URLs as Googlebot and noticed that it ...
1
vote
1answer
35 views

Can Google crawl .TS (MPEG2-TS) video format?

We are using the tool mp4hls (https://www.bento4.com/developers/hls/) to prep the hls stream. Basically, I think it can conver any video format to .mp4 then it convert to .ts. However, Google only ...
1
vote
1answer
35 views

How to get Google to crawl our product pages as soon as they are updated for in-stock or out-of-stock?

We have an automobile website where we have different cars and bike models. We have product pages for every car/bike. Each page has the products price, Offers, reviews , news, images and videos. ...
1
vote
1answer
46 views

Google Bot Crawls using expired cookies and fails to get response

We are using a cookie on our website which is to fetch some information through an API. The cookie becomes invalid after a very small duration. Google Bot uses the same cookie again and again to ...
1
vote
0answers
30 views

Does Google index ajax data that load after 1 second by setTimeout?

I use load Javascrpt and use Ajax for load content. And i should use setTimeout for run Ajax load function after 1 or 2 second(s). Does Google index and follow this content?
1
vote
0answers
13 views

Download link expired, going back to software presentation page, what redirect method to use?

When someone (user or bot) reaches an old download link like /download/yellow-wallpaper.zip?key=some-expired-download-key , user is sent back to file details page, e.g /yellow-wallpaper.html, where ...
1
vote
1answer
21 views

SEO - Custom error pages & status codes

My website uses a custom error page with the <meta name="robots" content="noindex"> meta tag. Am I right in thinking this tag will tell crawlers, such as Googlebot, to remove the page from ...
1
vote
1answer
67 views

Organic search traffic - big discrepancy between users and new users

Background: We are a non profit site only really acquiring traffic through paid ads. Our SEO is only just getting started so we don't get a high level of organic traffic. Recently we've seen an ...
1
vote
1answer
26 views

Alternative languages redirecting

Serving a page in English, EG: https://www.example.com/en/make-games Has the following meta tags: <meta rel="alternate" href="https://www.example.com/fr/make-games" hreflang="fr" /> <meta ...
1
vote
1answer
36 views

301 redirect not seen by googlebot

I have made a function on a wordpress site. The functions purpose is the following. 1) if the post is not found, redirect to the archive of the post. Should be fairly easy right? Well - according ...
2
votes
0answers
61 views

isAccessibleForFree and paywalled content delivered to Googlebots

I've got a site with some paywalled articles. We want to ensure that Googlebots can index the paywalled content, but we don't want to be accused of cloaking. I've got two types of articles: Premium ...
2
votes
2answers
175 views

Ampersand (&) in actual URL and sitemap

I have a sitemap (which is submitted to Google for indexing) in which I am adding URLs which have an ampersand (&) in it. Since in sitemap & has to be escaped I replaced & with &amp; ...
1
vote
1answer
192 views

Redirected page need to make no-index on google

All images used in WordPress website have got a URL and WordPress automatically created a URL for a landing page using the image.php file. So protecting to index all images with dedicated pages URL I ...
2
votes
3answers
63 views

Is Google Bot indexing the content of a textarea?

For one of my projects that provides pages with different content in a textarea, I need to make sure that Google Bot is indexing the content of those textareas. I could not find an answer online. ...
1
vote
1answer
243 views

Google started to reports root url blocked by robots.txt - but it's not -

My website has been running for multiple years without problem, however I recently noticed my root url on Google does not show any description/title. The Webmaster Tool reports this error: Crawl ...
1
vote
0answers
10 views

Why the Download Pages of our products are not indexed by Google? [duplicate]

I have a question about pages indexed, hope you can help me out, thank you in advance. Recently, we found our French site's two download pages are not indexed by Google. The URL of the French site ...
1
vote
0answers
170 views

Security Issue - Harmful content

I am trying to solve an issue with my website already for some time but without any luck. Google has detected somewhere in my website harmful content (Malware). But the problem is, I could not find ...
1
vote
0answers
70 views

Angular2 - Google indexing loader instead of redirected home page

I'm working on a multilanguage app in Angular. Universal is already setup and working but I have issue with googlebot indexing the page. When user navigates to url https://example.com only loader ...
5
votes
1answer
126 views

Does Google return to visit a page with 404 Error?

If a page indexed in Google shows a 404 Error, does Google return to visit it again? In other words, if a page that was indexed in Google continue to show the 404 Error for several years, will ...
3
votes
1answer
34 views

Status code vs. Response Header Status: what is the rule for bots?

Assuming we have a page, which has as Status Code 200 OK and as Response Headers Status HTTP/1.1 410 Gone, like on image: What is the behavior of bots / Googlebot? To crawl/index or not?
1
vote
1answer
69 views

Googlebot refuses to render more than index.html in Ember app

I am using Ember 3.0.0 to create a booking site. Title and meta tags are handled via ember-cli-meta-tags and seem to be rendered correctly when pages are opened in the web browser. However, Googlebot ...
1
vote
2answers
202 views

Does Google's own bot contribute to my Google Maps API fees?

Does Googlebot, when crawling pages which display maps using the Javascript API, cause a billable API usage on each page request?
0
votes
2answers
81 views

Stop Google from accsesing pages without robots.txt

Is it possible to get Google to stop crawling certain pages without using robots.txt? My understanding is robots.txt can block Google from accsesing a page, but if using noindex (meta or header) ...
1
vote
0answers
33 views

Could using the Google URL removal tool for a specific page also have removed the home page?

I noticed today my Homepage URL is not on Google Search Result anymore. Firstly for specific keyword where i ranked #1 than for domain itself. it is my brand keyword which I get most traffic from. I ...
1
vote
1answer
70 views

Sitewide A/B or Sanity Test

We are switching to a new Wordpress template and need some help. We will monitor if the test version improves the sites performance (CTR, Revenue) Technically we will: Split A Records by 50% for ...