Googlebot is the bot software that Google uses to crawl over 20 billion pages each day, the data obtained during a crawl is then analyzed and ranked on Google Search.

learn more… | top users | synonyms (1)

0
votes
2answers
74 views

Should I add the WWW version of my site to Google Webmaster Tools or Non-WWW or both?

I've found here similar topics but not exactly similar and without satisfying references - I'm going to launch a new domain, I understood it's better to have one pattern - whether www or non-www, so ...
0
votes
2answers
52 views

Lynx shows nothing but “loading” when opening a site

The site I help to administer appears very poorly in Google search results. While trying to tackle the issue from several angles I've tried to open the site in Lynx browser (the intention was to see ...
0
votes
2answers
135 views

Will robots.txt block crawling things like menu and footer links?

If I am blocking a page called pay.php, google won't crawl this page. But what about the links that are on this page as default as menu and footer links? Will these links also be blocked from ...
0
votes
1answer
79 views

Facelets SEO advice

Before I deploy the new version of my site, I would like to know a little more about facelets and SEO. My previous version of site was written in JSF and I noticed that when I submitted it to the ...
0
votes
2answers
33 views

Does Googlebot (and/or search engines) index a forwarded page? [duplicate]

Let's say I have example.com domain, and I force the user to use the HTTPS over HTTP. The question is as browsers just accept and load the forwarded/new page (when the request for http://example.com ...
0
votes
2answers
51 views

SEO: does Google index search result pages?

I have a site with a search form. When someone searches, the url for the results page has PHP variables in it and looks like: ...
0
votes
1answer
27 views

Putting a website in a directory above the public_html,Can search engines find it?,Is this website crawled by search engines?,What is the solution?

I have a Linux web host and I have added some other domains beside the main domains,as add-ons domains. I have created a directory beyond the main public_html directory, for each of them,and all of ...
0
votes
2answers
45 views

Google Webmaster Tools crawl errors due to Google decoding links that were urlencoded by Codeigniter

In our pages we have internal links that are urlencoded because the URL may sometime contain special characters such as () or ! So in our source code we have a link: ...
0
votes
1answer
94 views

How do I find when Googlebot will visit(crawl) my pages again?

I implemented the unavailable_after tag in my pages like the below and set to unavailable after 3 days but in Google Webmaster Tools I got 404 for that page. <META NAME="GOOGLEBOT" ...
0
votes
1answer
91 views

Is it safe to manipulate with DOM (using JavaScript) to move up in the search results?

I have some text written for SEO purposes and it is located right after the opening body tag, so it is at the top of the page. That doesn't look great so this text is moved to the bottom of the page ...
0
votes
1answer
75 views

Problem with Google crawler and cached results

I have a client that has 3 domain names. For the sake of this argument I will call them www.website1.com, www.website2.com and www.website3.com. He used to have a website package with VistaPrint and ...
0
votes
1answer
74 views

Why did the Googlebot add “sp=1” to my URL's?

Googlebot crawled my site for the first time today, and added something strange. It appended &sp=1 to my URL's. It seems like it's trying to gain authorized access? Normally I have ...
0
votes
2answers
147 views

Hide elements for Google

I have some elements which I don't want Google to see. Usually repetitive content, e.g. header and sidebar. As Googlebot doesn't handle AJAX requests, I load these resources with AJAX and insert them ...
0
votes
2answers
129 views

Anyone heard of 'tank tracking'?

Someone I know told me about some seriously cutting edge blackhat SEO that he called 'tank tracking'. He said that it is some sort of code (he believes written in Python) that 'sits' around the ...
0
votes
1answer
235 views

Google AdSense bot's algorithm and behavior [closed]

I am interesting in Google AdSense bot's algorithm and behavior with web site. I did not work with AdSense and I do not have account. So I need your help to understand: 1) Google's bot from time to ...
0
votes
2answers
55 views

Drop in traffic but increase in pages crawled per day

I saw a drop in traffic of my website in early September 2014. But, if I check the "Crawl Stats" in Google Webmaster Tools, I see an increase in "Pages Crawled per day" and "Kilobytes downloaded ...
0
votes
1answer
20 views

Google wont apply adsense policies for session based websites and Ajax loaded sites?

I have seen so many websites (Mostly User generated content - Getting contents from Third party API's) still displaying Adult like contents (with images) but still they are running well with Google ...
0
votes
1answer
45 views

Can Googlebot read CamelCase file names?

This has been a question I have been wondering for quite some time now. Can Googlebot, and other search engine crawlers, read file names (such as images) that are in CamelCase? For example, if I ...
0
votes
1answer
36 views

How to see my site's problems with Google?

I recently made a website: Boun Schedule Planner, which aims for the search query "Boun Schedule Planner" It was like 5th or 6th for a few days. But now it's not even in the first 3 pages. There are ...
0
votes
1answer
75 views

Remove Lots of index pages from google bot from same directory.

My question is about Google bot, seo and webmaster. may be this is not right place for ask, but if person not know so where he place his question he doesn't know!!! If you know my question's answer ...
0
votes
1answer
228 views

Will using page headings (h1, h2, h3 etc) as internal links encourage Google sitelinks? [duplicate]

I am trying to change my site's page headings to better emphasize the structure of the page's content hierarchy. I am hoping this will encourage Google Sitelinks and list snippets. Current headings ...
0
votes
1answer
35 views

Why does my htm pages have a number attached at the end of the link?

When I wanted to fetch a certain page with htm extension I noticed an attachment after the htm extenstion http://www.example.com/IP_814.htm#.Uk1_MtKsh8E Google could not fetch this page it gave the ...
0
votes
1answer
104 views

Need Google to recrawl the pages

I've changed the meta description tag in many of my site pages. Should I select Fetch as Google->Fetch->Add to Index->URL and all linked pages in GWT? Will Google recrawl the whole site faster than ...
0
votes
2answers
178 views

Does Submit to Index on a page with new content update Content Keywords for the site?

Using Google Webmaster Tools I'm trying to update the Content Keywords of my site. I'm confused about the relationship between Submit to Index and Content Keywords Does Fetch as Google --> Submit to ...
0
votes
2answers
2k views

How long does it take for Google to remove page from index? Or, why aren't these pages excluded?

A set of pages are marked as noindex and nofollow, both in robots.txt and with X-Robots-Tag: noindex, nofollow When checking with Google Webmaster Tools, the pages are reported as "Denied by ...
0
votes
2answers
91 views

Google not crawling my website [duplicate]

I own a website http://citrusbug.com. We did not have proper content before so to prevent search engine from crawling our site we added a robots.txt with code to prevent crawling. Now we had ...
0
votes
1answer
84 views

How can I create a sitemap for 2000 products powered by a single php page?

Usually I generate sitemaps using www.xml-sitemaps.com and the limit is 500 pages. My website has barely 30, but I have a lot of products in my database, so the page product.php becomes ...
0
votes
1answer
58 views

How does Google crawler find pages?

Will Google crawler find a page that is not linked to? I.e. pages accessible by manually modifying the url, like http://www.site.com/node/1/[secret keyword]. Note that it is only secret in that there ...
0
votes
1answer
67 views

What time does Googlebot crawl sites

Daily evening I sync the localhost copy of website to the production one. Yesterday I could not do it and I did so today at 10:00 am. Common sense says that Googlebot should be crawling the site when ...
0
votes
1answer
31 views

How do search engines treat out-tracked links

Some websites have out tracking / outgoing click tracking so that when you click on a link such as mysite.com on example.com you momentarily get taken to example.com/outgoing/mysite.com which then ...
0
votes
2answers
104 views

Googlebot ignore my new site [duplicate]

I have the following situation: I had a blog (made using Joomla) that in some month was pretty indexed by Google. For some technical problem I have delete it and I have newly created using WordPress ...
0
votes
4answers
249 views

Hide your website links from search engne

My Website Link is posted in someone else website, i asked for removal but can't get reply, google crawl his website and my website link is posted on that how to control that is that is possible? i ...
0
votes
2answers
575 views

When using Mod_rewrite does google bot see the original url or the rewritten url?

If i use the bellow code in my .htacess file would google bot (or any other search spiders) see the example.tumblr.com url or would they see the same url as the user ie. example.co.uk/blog ? ...
0
votes
1answer
412 views

How does Google feel about overflow: auto?

I am planning to add some keyword rich text to my homepage at the very top. I have written almost a mini essay and putting all this text at the very top left of my page wouldn't look too nice. I was ...
0
votes
1answer
98 views

Specifying crawlers to not crawl Links which are dependent on external API's

I have links on my site which lead to internal pages that depend on first bringing data from an external API. This only takes time the first time they are pushed. Links to pages which exists in the ...
0
votes
1answer
19 views

404 code/header for search engines, on removed user content?

I just got an email, from a former user on my website He was complaining that Google still shows the contact page he created on my site, even though he deleted it a month ago This is the first time ...
0
votes
0answers
15 views

Searching for Googlebot in server logs [closed]

I'm trying to use Heorku Logentries to track Googlebot's activity on my server. (Although, I think this could broadly apply to tracking Googlebot activity on any server.) I can see that I have results ...
0
votes
0answers
8 views

Links removed from google search results [duplicate]

For my website www.trickwarehouse.com i have created one post on keyloggers. When i search in google that what are keyloggers? then my website's post position was 16 but after 1 hour, when i checked ...
0
votes
1answer
62 views

google webmaster tools, site maps and indexing, number of pages indexed reduces with no new sitemap

It seems to take two weeks ish to get updates on to the google index, which seems fair enough as I imagine googles index of the whole internet could be rather large. However, the some changes to my ...
0
votes
1answer
107 views

Is it a problem for Google if robots.txt blocks URLs in the sitemap?

I had submitted my site's URL to Google Webmaster Tools. I am using OpenCart. I also have robots.txt file which is accessible to Googlebot. Now issue I am facing: Googlebot is blocked from accessing ...
0
votes
1answer
98 views

How to get Google to index dynamic pages generated from user IP address

I am creating a local community directory site which identifies the location of the user by IP address and dynamically renders the content related to the specific community identified by the IP ...
0
votes
0answers
43 views

Googlebot can't access my site

For some reason, Google Webmaster Tools started showing all these errors while crawling my site, saying that it couldn't reach my pages. I checked my robots.txt everything checks out, so I'm not sure ...
0
votes
0answers
46 views

How can I stop Googlebot can't access your site messages from a site that's moved to another developer

I had a client who moved their site a while back to another developer. When they moved it they took down the site and put up a coming soon page ( which is currently what's there). Googlebot warned me ...
0
votes
1answer
103 views

Several <a href=“#”>link</a> on one page - Effect on SEO and User Experience?

I've been researching the question if several <a href=“#”>link</a> links on one page have a bad impact on SEO and User Experience, for quite a while now but didn't really come to a ...
0
votes
0answers
11 views

Can google crawl data on googlemap?

I have a web service that includes mainly data on map. Can google fetch these data (asynchronously loaded )? the solution I'm opting right now is to set the data urls in a regular sitemap but there ...
0
votes
0answers
57 views

Why does Google keep hitting only one MP3 file?

I have a bunch of Mp3 files on my site, and all if them are in the sitemap to allow search engines that log mp3 files to know about them. For some reason, every day or two, Google keeps hitting the ...
0
votes
2answers
51 views

Question about external JavaScript and SEO [duplicate]

I have a question about external JavaScript and SEO. Assume I have a site called example.com with and index page of: <html> <body> <div class='main-container'> <script ...
0
votes
0answers
29 views

Sudden decrease in number of pages indexed in Google [duplicate]

One week ago, I updated the sitemap and pinged Google and other search engines to alert them to the change. However, the site now has lost like around 50 indexed pages (yes the site is my personal ...
0
votes
0answers
8 views

Disabling Single Event and 301 Redirect [duplicate]

Is it bad practice to have a 301 redirect for an event that has been disabled or deleted from a system to the list view? or should a 404 page? I dont want Google to ban me or anything.
0
votes
0answers
12 views

How do services that make subsites (ie. new subdomains) get those sites indexed by Google and other search enginex? [duplicate]

Services like tumblr, github pages, quora blogs, etc. make new subdomains when a user signs up or makes a site. So the subdomains are like: blogname.tumblr.com , pagename.github.io, ...