Googlebot is the bot software that Google uses to crawl over 20 billion pages each day, the data obtained during a crawl is then analyzed and ranked on Google Search.

learn more… | top users | synonyms (1)

0
votes
1answer
13 views

Issuu - PDF - Duplicated Content

I often write newsletters and publish them on my own site as PDF's. To potentially drive more traffic to my site, I was thinking about publishing the same newsletter files through sites like Scribd ...
0
votes
0answers
11 views

Odd requests from AdsBot-Google which re-include the domain name

It appears that since we started an AdWords campaign, our servers get tons of bogus requests from AdsBot-Google that for some reason include our own domain name in them. Example, if our website has ...
0
votes
2answers
27 views

Some questions: about google webmaster tool

I have a site like "example.com". Questions 1: I can access to my site through "example.com" and www.example.com in web browser. Both non www and www version redirects to example.com. So, is it ...
1
vote
1answer
70 views

How to serve for International Googlebots if we redirect users to subdomains for language matching?

So, we have a site and a few sub-domains with content in different languages and we're using Hreflang and canonical implementations for SEO. We do detect the language that users use in their browsers ...
0
votes
1answer
7 views

Submission of nested sitemaps and nesting levels

I'm considering nested site maps for a site with many small files. I'm thinking of nesting sitemaps going 3 levels deep into the directories. Question 1: When you submit nested sitemaps to Google: ...
0
votes
0answers
52 views

redirecting bots and crawlers to another but not human via htaccess

I would to apply this diagram via htaccess I tried a lots of codes but I failed every time so I need to redirecting bots and crawlers especially from facebook via htaccess regards
0
votes
1answer
23 views

How to control old indexed page crawling rate

I have a site where user can create their profile page with their specific URL. Each specific URL will be available to public which means that it will also be ready for search engines too to crawl. ...
0
votes
3answers
32 views

Strategy for mobile redirection and using alternative and canonical links

I had a separate mobile site uploaded so as to make Google happy. My big problem is that I had a desktop page that was once showing in the top 3 results on Google, and now it and its mobile ...
1
vote
1answer
42 views

Fetch as Google fails in Search Console (Webmaster Tools)

Most of the time, the Fetch as Google succeeds only partially. There's always some resources that cannot be retrieved. It says the URLs are temporarily unreachable. I thought it was something to do ...
2
votes
1answer
58 views

Is google penalizing SEO of websites with non device-width viewports?

These articles make it look like they are: https://support.google.com/webmasters/answer/6101188?hl=en https://developers.google.com/web/fundamentals/layouts/rwd-fundamentals/set-the-viewport This ...
0
votes
2answers
61 views

Googlebot couldn't get all resources for this page

When I fetch and render it is showing partial. It looks fine. The same way Google sees it is the same way visitors see it. The reason for this is that I've added BuySellaAd codes in a widget onto my ...
1
vote
1answer
50 views

When will google report correct results with its fetch tool?

I visited Google Webmaster Tools today after resubmitting sitemaps for my website, and while google is happy to index more HTML pages of my site, it however has not indexed a single image yet. So I ...
4
votes
0answers
552 views

How Googlebot discovers new website? [duplicate]

I have purchased a domain and uploaded with HTML Page and When I search in Google it shows my website. Now my question is How Google will discover my new website? I have not used ...
0
votes
0answers
20 views

Google Bot Crawler Issue with Codeigniter URLS

I received 6000+ crawler errors in my webmaster tools account. following is the one of the URL that identified as problematic URL. ...
0
votes
1answer
47 views

Why does Google Cache break my iframes?

In HTML5, iframe comes as a replacement for frameset. However, when I load my iframe-based page in SeaMonkey through Google Cache, the page appears empty, other than the standard Google Cache header ...
3
votes
1answer
55 views

Replace all images when search engine crawler visits site

I have written code that will detect when a user is a search engine and i dont include css files when this happens to save bandwidth. I was also thinking i could replace ALL images on the page with a ...
2
votes
2answers
50 views

Does Google index images declared using 'srcset'?

As of Chrome 40, the srcset attribute is supported, but will Google index the images within it?
0
votes
0answers
32 views

How post or page of facebook, twitter, forum or classified sites (HIGH PR) show instantally in google or other search engines

My question is about SEO. How post or page of facebook, twitter, forum or classified sites(HIGH PR) show instantally in google or other search engines. As i know google take few days for crawl. Then ...
1
vote
0answers
67 views

Googlebot follows rel=“nofollow” links

I would like to hide all pages of our website from being read or indexed by Googlebot, except several thousand of pages which were selected for indexing. The site was launched in April 2015. The ...
0
votes
0answers
49 views

google displays wrong language on multi language website because it takes cookies into consideration

I have a multiple language website that uses both subdomains and cookies to display the right language. Here is the logic on the website: de subdomain will display the German version fr subdomain ...
0
votes
0answers
12 views

Remove all pages from site [duplicate]

I have a staging server that accidentally got indexed by google. I want to remove ALL pages from google for that site: i.e. stagingserver.mycompany.com. I added a robots.txt to the root of the site ...
1
vote
1answer
90 views

For how long does the google bot wait after document ready before taking a snapshot of the webpage?

As the google bot lets the JS do its work on the page while crawling, for how long does it wait to let the javascript execute after document ready ? Suppose I have an AJAX response that takes longer ...
0
votes
1answer
34 views

Changing pagination, and having more sections per page

I'm upgrading an old website, from having 25 results per page (40 pages), to having 50-100 per page (5-10 pages) This will change the pagination, and maybe affect search engines in a bad way.. So I'm ...
3
votes
1answer
109 views

Why is the date displayed in the Google search result is 3 days older than the actual publication date?

Yesterday I published an article on my website, if I go on Google it says that it has been published 3 days ago. Why? I've WordPress, on a Centos 6,5 VM. System time is wrong of 20 min of my server, ...
3
votes
1answer
95 views

Does every page HAVE to be mobile friendly even if its almost unimportant?

I received the following regarding a URL of a page recommending a feature that only applies to desktop users: Faulty Redirect URLs redirect to an irrelevant landing page instead of the ...
1
vote
1answer
62 views

Fetch as Google Bot not working for `www.` version of site but is for `non-www.` version - Webmaster Tools

Ive recently acquired a new domain that i want to point at our existing domain, ive setup hosting for the new domain and then done a 301 redirect in the .htaccess file of the new domain to my existing ...
0
votes
1answer
15 views

Testing server indexed in Google [duplicate]

Assume our website domain to be www.example.com. And we have created a separate testing environment within our main domain: Say : www.project.example.com. So we have two versions within our main ...
0
votes
2answers
47 views

Soft 404 for smartphones

We have redirected all non-existing pages to home page. It was fine for some days. Few days back i saw increase in the number of soft 404 errors for desktop. And the errors were the page urls that ...
0
votes
2answers
59 views

can google index/crawl random subdomains that aren't linked to from anywhere

Me and a friend are having a debate where she claims that every and all subdomains will get crawled/indexed by google unless you specifically tell it not to, and I'm saying that if a page isn't linked ...
1
vote
1answer
36 views

How does an adsense bot determine the “Above the fold” space on mobile and desktop sites?

I ran an experiment last night by replacing a small ad for a mobile site thats located near the top of the page with a larger box ad, but I moved that ad towards the bottom of the page. Ten hours ...
5
votes
3answers
131 views

For webpages using jQuery, will the Googlebot index them on “page load” or after the “page ready” state?

Will the Googlebot index a webpage on "page load" or after the "page ready" state. The reason I'm asking is because I'm trying to change my h1 and h2 tags using jQuery on page Load.
1
vote
0answers
9 views

proper use google webmaster tools URL Parameters [duplicate]

There are some urls like http://domain.com/user/:username and http://domain.com/post/:id on my website, the username and id is the parameter from database. I try to let google crawler more efficiently ...
4
votes
3answers
200 views

How do I prevent Google from indexing text that is generated by JavaScript?

Google is executing the JavaScript on my password creator website and indexing the random passwords that get generated by the JavaScript. The random text that is generated could look spammy to ...
4
votes
1answer
65 views

Does Google now factor the prominence of how a link is rendered into their PageRank algorithm?

Google's John Mueller now says that Googlebot renders the pages it crawls to determine what content is on them. They do this because so many sites are now heavily JavaScript centric and this allows ...
1
vote
3answers
364 views

Is this really GoogleBot, or is this a spammer?

I found this IP in my access log posting repeatedly thousands of times. Is this really GoogleBot, or is it a spammer? I hesitate to block it. 185.62.188.98 - - [13/May/2015:18:54:33 -0400] "POST ...
1
vote
2answers
45 views

Duplicate Sitemap URLs

I'm in the process of replacing my old sitemaps with a new form which should better represent our site. However, as bots can take days or even weeks to scan all of our urls after submitted I do not ...
3
votes
3answers
84 views

Facets: Googlebot encountered extremely large numbers of links on your site

We are encountering the following issue: Googlebot encountered extremely large numbers of links on your site. The site is an ecommerce site and has around 12M pages indexed in Google. From the ...
3
votes
1answer
172 views

Webmaster Tools shows URL errors linked from http://com.example.www/:http

I have a bog standard WordPress 4.2.1 site. I recently added it to Webmaster Tools and it is showing me URL errors (404s) that are linked from a weird URL: http://com.example.www/:http (I have ...
2
votes
1answer
51 views

rel=“prev” and rel=“next” inserting using javascript

does adding the rel="next" tag after the page is loaded with a small javascript still useful for SEO and google? i have a problem in my code, basically the total number of pages in the catalog is ...
0
votes
0answers
9 views

Block search engine from indexing Currency Words as content keywords [duplicate]

I have one website with currency converter on it and Google WMT is considering the currency words as Content keywords. Is there anyway to block search engine from indexing this Currency Words like ...
0
votes
1answer
70 views

Am I doing .htaccess right to block google, bing and alexa bots?

I do not need bots at all. I bought my traffic and I do not want my sites to show up on search engine or anything. So I added this: <IfModule mod_setenvif.c> # remove most popular bot ...
1
vote
0answers
177 views

Google Webmaster Tools can't access robots.txt and the home page is “temporarily unreachable”

My site http://goo.gl/xH2Ffy suddenly disappeared from Google search and Webmaster Tools says it can't access robots.txt. Fetch as Google says "temporarily unreachable" for robots.txt and even for the ...
1
vote
1answer
449 views

Webmaster Tools “Fetch as Google” doesn't fetch my js/css/images on a CDN

I've discovered that when I do Google Webmaster Tools --> Fetch as Google on a page on my site (let's call it http://example.com), it shows the static assets (js, css, and images) as Temporarily ...
2
votes
1answer
266 views

Using Fetch as Google on AngularJS

I am working on an eCommerce site and I have been asked to look at the SEO implications of our product listing page. I have done all the necessary steps to get the webmaster tools working fine on one ...
1
vote
2answers
119 views

SEO on AJAX based search site

I'm building a website which is basically a simple 1 page with a search box, and AJAX'd search results displayed in a table. I'm having trouble finding out the correct information about crawlability ...
0
votes
0answers
7 views

Why is googlebot trying to fetch click.php? [duplicate]

I see the following in my access log: access.log.1:66.249.69.81 - - [02/Apr/2015:23:03:13 -0400] "GET /click.php?c%3Ec4C%7B%5BHG%7BbD6kc31%3E'dmjdl(..more such snipped..) HTTP/1.1" 404 19 "-" ...
3
votes
3answers
403 views

How to make Google index millions of user profile pages?

I have a website with a lot of user profiles (like LinkedIn or the like). I need Google to index each public user profile page. I have a few things on my mind: 1- There is no user listing on my ...
1
vote
0answers
48 views

Google ate all my bandwidth [closed]

Alright, I am new here and need some good help. I am running an e commerce build on magento called zumastic.com. The issue, I am facing is that google is eating a huge amount of my bandwidth. Last ...
8
votes
4answers
754 views

Google doesn't crawl CDN files

I've noticed that Google Webmaster Tools is reporting a lot of blocked resources in my website. Right now all the "blocked resources" are .css, .js and images (.jpg, .png) that I serve from Cloudfront ...
1
vote
2answers
263 views

Conflicting test results when using The Mobile-Friendly Test and PageSpeed Insights

Our website uses an open source product to detect user agent and based on that, it servers mobile theme or desktop theme to users. I know the exact difference between mobile-friendly test and ...