Stack Exchange Network

Stack Exchange network consists of 174 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [googlebot]

Googlebot is the bot software that Google uses to crawl over 20 billion pages each day, the data obtained during a crawl is then analyzed and ranked on Google Search.

2
votes
2answers
36 views

Do non-existent 'category' pages in URLs present a problem for search engines?

Say for example if I have a news site and my URL structure is: example.com/news/random-article If there is no page at example.com/news/, in other words it results in a 404 error, is that problematic ...
0
votes
0answers
15 views

My website's name doesn't appear on google [duplicate]

My website has been online for 10 days. (content complete) I've allowed Google boots. I also sent a sitemap to the search console. (I use yoast seo by the way) In this 10 days pages were added to ...
3
votes
0answers
29 views

Google not showing cache option on the following url in SERP results?

Google is not showing option to check cache on the following url that I have mentioned below in SERP results.
-1
votes
0answers
11 views

Why my website just homepage title not updating on google? [duplicate]

I have the issue on my website Essays Help I fetch as google and I wait around a week but my website meta title isn't updating on google please help me? website: My Essays Help
0
votes
0answers
14 views

Dramatic increase in time spent downloading a page and barely no pages crawled per day

For the past 2 months, Google has been barely been crawling our website while the increase in time spent downloading a page has gone up tremendously. Our website is fast and optimized, on average 77 ...
2
votes
1answer
20 views

Can I use robots.txt to allow Google to access some folders, but disallow everything else?

I want to allow Google bots to crawl only my index.php and the content of some folders. Would that work? User-agent: * Allow: /index*.php Allow: /folder1/ Allow: /folder2/ Allow: /folder3/ Allow: /...
1
vote
1answer
42 views

Why are some google snippet results bold? [on hold]

Recently, when I search for keyword "best mlm software" I got a featured snippet having "List of Multi Level Marketing Software". I noted that two of list's item were BOLD text. Didn't know why??? Can ...
1
vote
1answer
23 views

Does Google Crawl Rate have an impact on Google's Image Bot

I have a relatively large website. Each page has several images. Google indexes my website very slowly, but it actively crawls my pictures. Does changing the Google Crawl Rate have an impact on Google'...
2
votes
1answer
52 views

Is it good for SEO to change an article's slug when changing its title?

I have the following URL view: example.com/article/{slug}-{id} If I change the title of the article, then the slug will also change so that there is no disagreement. Please note that the article ...
2
votes
2answers
30 views

How many levels of sub-pages do Google bots crawl to when fetching and indexing linked pages in Google Search Console?

Let's say I submit a sitemap page that contains all the links I want Google to crawl, but instead of submitting a sitemap, I fetch the page and linked pages and let Google index them. How many levels ...
2
votes
0answers
79 views

Discus comments cause “clickable elements too close together” in Google Search Console mobile usabality report

I just got an email from Google that they found some problems on my site: clickable elements too close together Content wider than screen I fetched problematic URLs as Googlebot and noticed that it ...
1
vote
1answer
27 views

Can Google crawl .TS (MPEG2-TS) video format?

We are using the tool mp4hls (https://www.bento4.com/developers/hls/) to prep the hls stream. Basically, I think it can conver any video format to .mp4 then it convert to .ts. However, Google only ...
1
vote
1answer
28 views

How to get Google to crawl our product pages as soon as they are updated for in-stock or out-of-stock?

We have an automobile website where we have different cars and bike models. We have product pages for every car/bike. Each page has the products price, Offers, reviews , news, images and videos. ...
1
vote
1answer
37 views

Google Bot Crawls using expired cookies and fails to get response

We are using a cookie on our website which is to fetch some information through an API. The cookie becomes invalid after a very small duration. Google Bot uses the same cookie again and again to ...
1
vote
0answers
26 views

Does Google index ajax data that load after 1 second by setTimeout?

I use load Javascrpt and use Ajax for load content. And i should use setTimeout for run Ajax load function after 1 or 2 second(s). Does Google index and follow this content?
1
vote
0answers
12 views

Download link expired, going back to software presentation page, what redirect method to use?

When someone (user or bot) reaches an old download link like /download/yellow-wallpaper.zip?key=some-expired-download-key , user is sent back to file details page, e.g /yellow-wallpaper.html, where ...
1
vote
1answer
19 views

SEO - Custom error pages & status codes

My website uses a custom error page with the <meta name="robots" content="noindex"> meta tag. Am I right in thinking this tag will tell crawlers, such as Googlebot, to remove the page from ...
1
vote
1answer
51 views

Organic search traffic - big discrepancy between users and new users

Background: We are a non profit site only really acquiring traffic through paid ads. Our SEO is only just getting started so we don't get a high level of organic traffic. Recently we've seen an ...
1
vote
1answer
24 views

Alternative languages redirecting

Serving a page in English, EG: https://www.example.com/en/make-games Has the following meta tags: <meta rel="alternate" href="https://www.example.com/fr/make-games" hreflang="fr" /> <meta ...
1
vote
1answer
30 views

301 redirect not seen by googlebot

I have made a function on a wordpress site. The functions purpose is the following. 1) if the post is not found, redirect to the archive of the post. Should be fairly easy right? Well - according ...
2
votes
0answers
39 views

isAccessibleForFree and paywalled content delivered to Googlebots

I've got a site with some paywalled articles. We want to ensure that Googlebots can index the paywalled content, but we don't want to be accused of cloaking. I've got two types of articles: Premium ...
2
votes
2answers
97 views

Ampersand (&) in actual URL and sitemap

I have a sitemap (which is submitted to Google for indexing) in which I am adding URLs which have an ampersand (&) in it. Since in sitemap & has to be escaped I replaced & with &amp; ...
1
vote
1answer
112 views

Redirected page need to make no-index on google

All images used in WordPress website have got a URL and WordPress automatically created a URL for a landing page using the image.php file. So protecting to index all images with dedicated pages URL I ...
2
votes
3answers
61 views

Is Google Bot indexing the content of a textarea?

For one of my projects that provides pages with different content in a textarea, I need to make sure that Google Bot is indexing the content of those textareas. I could not find an answer online. ...
1
vote
1answer
132 views

Google started to reports root url blocked by robots.txt - but it's not -

My website has been running for multiple years without problem, however I recently noticed my root url on Google does not show any description/title. The Webmaster Tool reports this error: Crawl ...
1
vote
0answers
10 views

Why the Download Pages of our products are not indexed by Google? [duplicate]

I have a question about pages indexed, hope you can help me out, thank you in advance. Recently, we found our French site's two download pages are not indexed by Google. The URL of the French site ...
1
vote
0answers
115 views

Security Issue - Harmful content

I am trying to solve an issue with my website already for some time but without any luck. Google has detected somewhere in my website harmful content (Malware). But the problem is, I could not find ...
0
votes
0answers
43 views

Angular2 - Google indexing loader instead of redirected home page

I'm working on a multilanguage app in Angular. Universal is already setup and working but I have issue with googlebot indexing the page. When user navigates to url https://example.com only loader ...
4
votes
1answer
91 views

Does Google return to visit a page with 404 Error?

If a page indexed in Google shows a 404 Error, does Google return to visit it again? In other words, if a page that was indexed in Google continue to show the 404 Error for several years, will ...
3
votes
1answer
31 views

Status code vs. Response Header Status: what is the rule for bots?

Assuming we have a page, which has as Status Code 200 OK and as Response Headers Status HTTP/1.1 410 Gone, like on image: What is the behavior of bots / Googlebot? To crawl/index or not?
1
vote
1answer
63 views

Googlebot refuses to render more than index.html in Ember app

I am using Ember 3.0.0 to create a booking site. Title and meta tags are handled via ember-cli-meta-tags and seem to be rendered correctly when pages are opened in the web browser. However, Googlebot ...
1
vote
2answers
136 views

Does Google's own bot contribute to my Google Maps API fees?

Does Googlebot, when crawling pages which display maps using the Javascript API, cause a billable API usage on each page request?
0
votes
2answers
76 views

Stop Google from accsesing pages without robots.txt

Is it possible to get Google to stop crawling certain pages without using robots.txt? My understanding is robots.txt can block Google from accsesing a page, but if using noindex (meta or header) ...
1
vote
0answers
31 views

Could using the Google URL removal tool for a specific page also have removed the home page?

I noticed today my Homepage URL is not on Google Search Result anymore. Firstly for specific keyword where i ranked #1 than for domain itself. it is my brand keyword which I get most traffic from. I ...
1
vote
1answer
67 views

Sitewide A/B or Sanity Test

We are switching to a new Wordpress template and need some help. We will monitor if the test version improves the sites performance (CTR, Revenue) Technically we will: Split A Records by 50% for ...
0
votes
1answer
15 views

Does Google take into account page title changes due to JavaScript? [duplicate]

One of my pages is set up so that the initial page title is just some placeholder title, and it changes after an AJAX call. Here's what the process looks like: Initial page load AJAX call to ...
1
vote
0answers
60 views

Stop getting traffic from Googlebot on high server load

In my website, we usually get a spike in the number of requests because of search engine crawlers (like Googlebot) harvesting the website for indexing. So what I thought of; is depending on the ...
1
vote
0answers
96 views

Google AdWords ads are disapproved because “destination not working”

We have been having an issue setting up adwords for one of our clients and despite multiple phone calls to Google we cannot get an answer out of them as to why our ads are being disapproved. ...
0
votes
0answers
20 views

How to manage multi-city or multi-area content within a country? [duplicate]

We have a gold jewellery website with 1000 products available in 500 cities. Based on the city- the price, offers, and physical stores are different for each product. Currently, we have 1 webpage ...
1
vote
2answers
58 views

SEO impact of changing versions (and names) of external CSS

A website of mine uses a WordPress plugin for CSS minification. The plugin regenerates the CSS file on every update to the style-sheet. When it does, the plugin also changes the version and hence name ...
0
votes
0answers
49 views

Relative URLs vs absolute URLs for SEO [duplicate]

This question has some fantastic answers on StackOverflow, especially from the dev perspective like this one https://stackoverflow.com/a/21828923/728236, but how about from an SEO perspective? There'...
1
vote
0answers
21 views

Product carousel on my site is showing up blank in Google's fetch and render for mobile

Using GSC, I am trying to fetch a page with products and content, this page also has few products in carousel format and when I render this page for desktop or mobile, I am seeing everything fine but ...
1
vote
0answers
18 views

What is the correct response to Googlebot for a search_term_string from site search markup with _escaped_fragment?

I have a AJAX web online-shop with hashbang URL and added SiteLinks as JSON-LD. I follow this introduction. { "@context":"http://schema.org", "@type":"WebSite", "url":"https://...
0
votes
0answers
21 views

Browser response vs crawler response: different HTML length

Because of length, Google and Yandex bots are saying to me that the page is not the same and that you are cloaking. But I do nothing about cloaking, here is the diff: Browser: <div class="...
0
votes
0answers
66 views

Seo Report Result Problems

This is my first seo project.I am new at SEO.Have some problems, Info : Webform ASP.NET 4.5 1-) "Accesible Index Page Configured wrong" error is my biggest problem.My site have 4 URL for homepage . ...
1
vote
0answers
10 views

My most important pages does not get indexed, google search console [duplicate]

I have a website with blog articles. I added it's sitemap to google search console and it say that the sitemap has been processed. However, half of my articles (the important pages) have not been ...
1
vote
0answers
13 views

Should a page with adsense be viewed at least once for adsense crawler to crawl?

I use plugins to insert Adsense codes on my WordPress website. The plugin has an option to disable ads for the admin user. Hence, I will not be seeing ads when creating and publishing/reviewing posts. ...
0
votes
0answers
14 views

header tags offscreen - punished by google? [duplicate]

If I put a few h2 and h3 tags with seo phrases on my site then used css or js to move them off the visible screen would googlebot be able to see that and would they punish me for doing so? There are ...
1
vote
0answers
34 views

Google cache is not showing up for a page despite trying twice to use the fetch and render tool [duplicate]

I did Fetch and Render twice on a particular page but the cached version of the page is still not showing up on Google. I did a Fetch and Render on other posts on the site and they all showed cached ...
0
votes
0answers
26 views

Any relation between adsense lack of impressions and incorrect sitemap?

I have recently submitted a sitemap in search console for at least 200 new pages on my site. However, I did a mistake by not including a slash ("/") at the end of each url. But the pages without ...