Googlebot is the bot software that Google uses to crawl over 20 billion pages each day, the data obtained during a crawl is then analyzed and ranked on Google Search.

learn more… | top users | synonyms (1)

1
vote
3answers
69 views

A/B Testing A Site Redesign Without Google Penalties

I am redesigning our job board website to be responsive. Rather than just replace the existing site with the responsive site, I want to A/B test the new responsive site alongside the existing site so ...
1
vote
3answers
241 views

Googlebot couldn't get all resources for this page

When I fetch and render it is showing partial. It looks fine. The same way Google sees it is the same way visitors see it. The reason for this is that I've added BuySellaAd codes in a widget onto my ...
1
vote
2answers
101 views

Should i noindex low/quality duplicate content tag pages?

Google have indexed 300k tag pages of my site, these are automatically generated based on the words on the title of each entry so it's likely to see pages which doesn't make much sense like: ...
3
votes
1answer
56 views

Should I remove Google indexed pages before requesting a review after thin content manual action

Following on from my previous question - NOINDEX large number of pages to fix Google thin content manual action? Firstly, If I noindex all pages causing the manual action, is that enough for me to ...
2
votes
1answer
73 views

When will Google report correct results with its fetch tool?

I visited Google Webmaster Tools(Google Search Console) today after resubmitting sitemaps for my website, and while Google is happy to index more HTML pages of my site, it however has not indexed a ...
0
votes
1answer
49 views

How google index the Ajax websites?

As I understand google depreciated its ajax crawling scheme they came up with back in 2009. I run single page app powered by Anguar frontend with push state navigation and my question is around ...
2
votes
0answers
188 views

Why is GoogleBot crawling HTML pages as JSON using as referrer the one from my android app?

Recently, I started finding in my logs errors caused by missing templates for pages that reasonably should have them, i.e. REQUEST_PATH : /foo/bar.json HTTP_HOST ...
1
vote
0answers
24 views

Does fetch and render + submit to index interrupt Google's crawl period?

If you resubmit your site to the index in Webmaster Tools, will pending crawls get cancelled? I ask because a client's site has been up for 72 hours now and has not been crawled. I submitted to index ...
1
vote
0answers
38 views

Does 'thin-content' content on some pages affect overall site SEO?

I have an UGC Q&A site. Due to it's nature is inevitable to see some 'thin' or low quality content in some posts. Will there be any sitewide impact in SEO for the other high quality pages? If yes ...
1
vote
0answers
41 views

Search engine ranking of paginated categories, subcategories and tag pages

Suppose we have an E-commerce domain with the following structure: domain/items - contains all items. domain/items/category - contains a narrower spectrum of results than above. ...
1
vote
0answers
53 views

How post or page of facebook, twitter, forum or classified sites (HIGH PR) show instantally in google or other search engines

My question is about SEO. How post or page of facebook, twitter, forum or classified sites(HIGH PR) show instantally in google or other search engines. As i know google take few days for crawl. Then ...
1
vote
0answers
98 views

Googlebot follows rel=“nofollow” links

I would like to hide all pages of our website from being read or indexed by Googlebot, except several thousand of pages which were selected for indexing. The site was launched in April 2015. The ...
1
vote
0answers
295 views

Google Webmaster Tools can't access robots.txt and the home page is “temporarily unreachable”

My site http://goo.gl/xH2Ffy suddenly disappeared from Google search and Webmaster Tools says it can't access robots.txt. Fetch as Google says "temporarily unreachable" for robots.txt and even for the ...
1
vote
0answers
98 views

How to handle URL encoded parameter separators when responding to AJAX crawled _escaped_fragment?

I've been reading about how to handle Google requested URL here There is an example from Google : Pretty URL: www.example.com?myquery#!key1=value1&key2=value2 Ugly URL: ...
1
vote
0answers
91 views

I still get “Googlebot can't access your site” messages after removing a site from Google Webmaster Tools

I had a client who moved their site a while back to another developer. When they moved it they took down the site and put up a coming soon page ( which is currently what's there). Googlebot warned me ...
1
vote
0answers
91 views

Why does Google keep hitting only one MP3 file?

I have a bunch of Mp3 files on my site, and all if them are in the sitemap to allow search engines that log mp3 files to know about them. For some reason, every day or two, Google keeps hitting the ...
1
vote
0answers
74 views

Suggest that Googlebot read “dezign” as “design”

I used the spelling "dezign" instead of "design" many times in my website -- even in my domain name. Is there any way to suggest that Googlebot read "dezign" word as "design" with correct spelling?
1
vote
0answers
64 views

Does Google bot always treat dot as word separator?

One of our products is a grid control called iGrid, and we have two versions for two different platforms. One is iGrid ActiveX, the other is iGrid for .NET Framework WinForms called "iGrid.NET". At ...
1
vote
0answers
200 views

Site returning 404 header to Google, not sure why

A Drupal site that works fine for regular users returns a 404 not found error when I try to use the W3C validator on it; it is also not being indexed by Google at all (which is the main issue but I ...
0
votes
0answers
42 views

Why is hidden content not always indexed?

I've read several blogs discussing the 'hidden content' issue. This article experimented with several options. It concludes that Google DOES index hidden content. The issues is also discussed here. ...
0
votes
0answers
95 views

SEO on ui-router states of AngularJS SPA

I have a simple SPA created with AngularJS, you can see it running here: www.consami.com.br. I was using Prerender.io to load a pure HTML version for crawlers, however some pages were not indexed by ...
0
votes
0answers
25 views

Crawl Budget economy: what to do with URLs with parameters?

So, a site has inbound links to a lot of important pages and these links have URL parameters. These URL parameters are basically used just for tracking purposes, similar to the way Amazon links to its ...
0
votes
0answers
23 views

access.log requests from UA “Google favicon”…? What do these mean?

I have a bunch of these in my nginx log file, on a range of URLs: on 66.102.9.23 - - [12/Nov/2015:10:40:25 +0000] "GET /myadminfolder/page.html HTTP/1.1" 200 1247 1800 "-" "Google favicon" ...
0
votes
0answers
23 views

How should I render a widget with many links inside a blog so google don't penalize the blog?

We run a service (will not mention the URL, as this is not a pitch) and need advice on Google-viewing-links matters. The service is for bloggers and allows them to create a "linkup" inside their blog ...
0
votes
0answers
41 views

Odd requests from AdsBot-Google which re-include the domain name

It appears that since we started an AdWords campaign, our servers get tons of bogus requests from AdsBot-Google that for some reason include our own domain name in them. Example, if our website has ...
0
votes
0answers
149 views

redirecting bots and crawlers to another but not human via htaccess

I would to apply this diagram via htaccess I tried a lots of codes but I failed every time so I need to redirecting bots and crawlers especially from facebook via htaccess regards
0
votes
0answers
72 views

Google Bot Crawler Issue with Codeigniter URLS

I received 6000+ crawler errors in my webmaster tools account. following is the one of the URL that identified as problematic URL. ...
0
votes
0answers
97 views

google displays wrong language on multi language website because it takes cookies into consideration

I have a multiple language website that uses both subdomains and cookies to display the right language. Here is the logic on the website: de subdomain will display the German version fr subdomain ...
0
votes
0answers
26 views

Google Crawler constantly reports ~2-3% errors

For the last year and a half, I have a constant 2-3% error rate while crawling my web site. Types of errors are "Server error: No response". I've switched ISP, but errors persist. Debugging of ...
0
votes
0answers
23 views

Question About Traffic From Google And GEO Location OF Website

I want to know lets say a person who lives in South Wimbledon, UK, runs a website and then outsource the website work (like writing articles and posting and other site management work etc..) to a ...