Googlebot is the bot software that Google uses to crawl over 20 billion pages each day, the data obtained during a crawl is then analyzed and ranked on Google Search.

learn more… | top users | synonyms (1)

0
votes
2answers
146 views

Will robots.txt block crawling things like menu and footer links?

If I am blocking a page called pay.php, google won't crawl this page. But what about the links that are on this page as default as menu and footer links? Will these links also be blocked from ...
0
votes
1answer
82 views

Facelets SEO advice

Before I deploy the new version of my site, I would like to know a little more about facelets and SEO. My previous version of site was written in JSF and I noticed that when I submitted it to the ...
0
votes
1answer
30 views

How to tell how long google thinks it has been since your website was updated?

I never really thought 'freshness' was a problem for my site until I recently used the silktide 'nibbler' website grading report / scanner. It listed my 'freshness' as a big old 0 and said my site ...
0
votes
1answer
16 views

google webmaster is crawling both file urls

webmaster is crawling both the urls, one which is SEO friendly and the one .php urls for the same page thus showing duplicate title and description for those pages. urls of .php were changed through ...
0
votes
2answers
61 views

SEO: does Google index search result pages?

I have a site with a search form. When someone searches, the url for the results page has PHP variables in it and looks like: ...
0
votes
1answer
34 views

Putting a website in a directory above the public_html,Can search engines find it?,Is this website crawled by search engines?,What is the solution?

I have a Linux web host and I have added some other domains beside the main domains,as add-ons domains. I have created a directory beyond the main public_html directory, for each of them,and all of ...
0
votes
2answers
136 views

Google Webmaster Tools crawl errors due to Google decoding links that were urlencoded by Codeigniter

In our pages we have internal links that are urlencoded because the URL may sometime contain special characters such as () or ! So in our source code we have a link: ...
0
votes
1answer
174 views

How do I find when Googlebot will visit(crawl) my pages again?

I implemented the unavailable_after tag in my pages like the below and set to unavailable after 3 days but in Google Webmaster Tools I got 404 for that page. <META NAME="GOOGLEBOT" ...
0
votes
1answer
109 views

Is it safe to manipulate with DOM (using JavaScript) to move up in the search results?

I have some text written for SEO purposes and it is located right after the opening body tag, so it is at the top of the page. That doesn't look great so this text is moved to the bottom of the page ...
0
votes
1answer
125 views

Need Google to recrawl the pages

I've changed the meta description tag in many of my site pages. Should I select Fetch as Google->Fetch->Add to Index->URL and all linked pages in GWT? Will Google recrawl the whole site faster than ...
0
votes
1answer
83 views

Problem with Google crawler and cached results

I have a client that has 3 domain names. For the sake of this argument I will call them www.website1.com, www.website2.com and www.website3.com. He used to have a website package with VistaPrint and ...
0
votes
1answer
75 views

Why did the Googlebot add “sp=1” to my URL's?

Googlebot crawled my site for the first time today, and added something strange. It appended &sp=1 to my URL's. It seems like it's trying to gain authorized access? Normally I have ...
0
votes
2answers
166 views

Hide elements for Google

I have some elements which I don't want Google to see. Usually repetitive content, e.g. header and sidebar. As Googlebot doesn't handle AJAX requests, I load these resources with AJAX and insert them ...
0
votes
2answers
135 views

Anyone heard of 'tank tracking'?

Someone I know told me about some seriously cutting edge blackhat SEO that he called 'tank tracking'. He said that it is some sort of code (he believes written in Python) that 'sits' around the ...
0
votes
1answer
239 views

Google AdSense bot's algorithm and behavior [closed]

I am interesting in Google AdSense bot's algorithm and behavior with web site. I did not work with AdSense and I do not have account. So I need your help to understand: 1) Google's bot from time to ...
0
votes
1answer
65 views

Does Googlebot see dynamically loaded content? [closed]

Our modelling agency website uses the Photolux Wordpress theme. Our front page uses the same masonry template that the demo site: Demo. The only difference is that we've modified the masonry elements ...
0
votes
1answer
69 views

Can Googlebot detect duplicate video content?

I downloaded a lot of videos from YouTube and uploaded them to my hosting. Can Googlebot-video can detect that my video is duplicate?
0
votes
2answers
265 views

Drop in traffic but increase in pages crawled per day

I saw a drop in traffic of my website in early September 2014. But, if I check the "Crawl Stats" in Google Webmaster Tools, I see an increase in "Pages Crawled per day" and "Kilobytes downloaded ...
0
votes
1answer
87 views

Can Googlebot read CamelCase file names?

This has been a question I have been wondering for quite some time now. Can Googlebot, and other search engine crawlers, read file names (such as images) that are in CamelCase? For example, if I ...
0
votes
1answer
52 views

How to see my site's problems with Google?

I recently made a website, which aims for the search query "Boun Schedule Planner" It was like 5th or 6th for a few days. But now it's not even in the first 3 pages. There are no warnings in ...
0
votes
1answer
114 views

Remove Lots of index pages from google bot from same directory.

My question is about Google bot, seo and webmaster. may be this is not right place for ask, but if person not know so where he place his question he doesn't know!!! If you know my question's answer ...
0
votes
1answer
370 views

Will using page headings (h1, h2, h3 etc) as internal links encourage Google sitelinks? [duplicate]

I am trying to change my site's page headings to better emphasize the structure of the page's content hierarchy. I am hoping this will encourage Google Sitelinks and list snippets. Current headings ...
0
votes
1answer
36 views

Why does my htm pages have a number attached at the end of the link?

When I wanted to fetch a certain page with htm extension I noticed an attachment after the htm extenstion http://www.example.com/IP_814.htm#.Uk1_MtKsh8E Google could not fetch this page it gave the ...
0
votes
2answers
292 views

Does Submit to Index on a page with new content update Content Keywords for the site?

Using Google Webmaster Tools I'm trying to update the Content Keywords of my site. I'm confused about the relationship between Submit to Index and Content Keywords Does Fetch as Google --> Submit to ...
0
votes
2answers
3k views

How long does it take for Google to remove page from index? Or, why aren't these pages excluded?

A set of pages are marked as noindex and nofollow, both in robots.txt and with X-Robots-Tag: noindex, nofollow When checking with Google Webmaster Tools, the pages are reported as "Denied by ...
0
votes
2answers
95 views

Google not crawling my website [duplicate]

I own a website http://citrusbug.com. We did not have proper content before so to prevent search engine from crawling our site we added a robots.txt with code to prevent crawling. Now we had ...
0
votes
1answer
100 views

How can I create a sitemap for 2000 products powered by a single php page?

Usually I generate sitemaps using www.xml-sitemaps.com and the limit is 500 pages. My website has barely 30, but I have a lot of products in my database, so the page product.php becomes ...
0
votes
1answer
63 views

How does Google crawler find pages?

Will Google crawler find a page that is not linked to? I.e. pages accessible by manually modifying the url, like http://www.site.com/node/1/[secret keyword]. Note that it is only secret in that there ...
0
votes
1answer
80 views

What time does Googlebot crawl sites

Daily evening I sync the localhost copy of website to the production one. Yesterday I could not do it and I did so today at 10:00 am. Common sense says that Googlebot should be crawling the site when ...
0
votes
1answer
33 views

How do search engines treat out-tracked links

Some websites have out tracking / outgoing click tracking so that when you click on a link such as mysite.com on example.com you momentarily get taken to example.com/outgoing/mysite.com which then ...
0
votes
2answers
120 views

Googlebot ignore my new site [duplicate]

I have the following situation: I had a blog (made using Joomla) that in some month was pretty indexed by Google. For some technical problem I have delete it and I have newly created using WordPress ...
0
votes
4answers
251 views

Hide your website links from search engne

My Website Link is posted in someone else website, i asked for removal but can't get reply, google crawl his website and my website link is posted on that how to control that is that is possible? i ...
0
votes
2answers
632 views

When using Mod_rewrite does google bot see the original url or the rewritten url?

If i use the bellow code in my .htacess file would google bot (or any other search spiders) see the example.tumblr.com url or would they see the same url as the user ie. example.co.uk/blog ? ...
0
votes
1answer
488 views

How does Google feel about overflow: auto?

I am planning to add some keyword rich text to my homepage at the very top. I have written almost a mini essay and putting all this text at the very top left of my page wouldn't look too nice. I was ...
0
votes
1answer
105 views

Specifying crawlers to not crawl Links which are dependent on external API's

I have links on my site which lead to internal pages that depend on first bringing data from an external API. This only takes time the first time they are pushed. Links to pages which exists in the ...
0
votes
0answers
6 views

Block search engine from indexing Currency Words as content keywords [duplicate]

I have one website with currency converter on it and Google WMT is considering the currency words as Content keywords. Is there anyway to block search engine from indexing this Currency Words like ...
0
votes
1answer
4 views

Am I doing .htaccess right to block google, bing and alexa bots?

I do not need bots at all. I bought my traffic and I do not want my sites to show up on search engine or anything. So I added this: <IfModule mod_setenvif.c> # remove most popular bot ...
0
votes
0answers
7 views

Why is googlebot trying to fetch click.php? [duplicate]

I see the following in my access log: access.log.1:66.249.69.81 - - [02/Apr/2015:23:03:13 -0400] "GET /click.php?c%3Ec4C%7B%5BHG%7BbD6kc31%3E'dmjdl(..more such snipped..) HTTP/1.1" 404 19 "-" ...
0
votes
0answers
16 views

Google Crawler constantly reports ~2-3% errors

For the last year and a half, I have a constant 2-3% error rate while crawling my web site. Types of errors are "Server error: No response". I've switched ISP, but errors persist. Debugging of ...
0
votes
0answers
13 views

Question About Traffic From Google And GEO Location OF Website

I want to know lets say a person who lives in South Wimbledon, UK, runs a website and then outsource the website work (like writing articles and posting and other site management work etc..) to a ...
0
votes
0answers
13 views

Trigger Google (re)crawl with ping [duplicate]

I have a site A with daily updated content. Right after the update, I send a webblog ping using various ping services, like http://rpc.pingomatic.com. Moments later, the updated homepage lands in the ...
0
votes
0answers
7 views

How to influence the info column in Google's search result? [duplicate]

Google often shows an info column next to the search results, if it recognizes your search query to be directly related to a "thing" they know. Example: You search for "beatles" and get a box with ...
0
votes
0answers
14 views

Site's pages hasn't been indexed for more than 10 days, why? [duplicate]

I checked out and robots.txt doesn't block the pages that I've been reviewing. The cached versions of the pages I reviewed in Google are made 10 days ago or more. What could be the cause that a site ...
0
votes
0answers
18 views

Fetch as google: Unreachable Error Message [duplicate]

A few days ago i changed my website CMS from vbulletin to MyBB 1.8.0 and all URLs structure changed, now I've encountering a very strange problem with "Fetch as Google"! Google can fetch my website ...
0
votes
0answers
21 views

Is cloaking your site for disqus comments good for SEO? [duplicate]

I use "disqus" as the commenting platform on my website. There is quite a bit of SEO relevance for the comment content but since its generated through JAVASCRIPT. I can't get the best of it. Can I use ...
0
votes
2answers
51 views

Does Googlebot (and/or search engines) index a forwarded page? [duplicate]

Let's say I have example.com domain, and I force the user to use the HTTPS over HTTP. The question is as browsers just accept and load the forwarded/new page (when the request for http://example.com ...
0
votes
1answer
26 views

404 code/header for search engines, on removed user content?

I just got an email, from a former user on my website He was complaining that Google still shows the contact page he created on my site, even though he deleted it a month ago This is the first time ...
0
votes
0answers
22 views

Searching for Googlebot in server logs [closed]

I'm trying to use Heorku Logentries to track Googlebot's activity on my server. (Although, I think this could broadly apply to tracking Googlebot activity on any server.) I can see that I have results ...
0
votes
0answers
8 views

Links removed from google search results [duplicate]

For my website www.trickwarehouse.com i have created one post on keyloggers. When i search in google that what are keyloggers? then my website's post position was 16 but after 1 hour, when i checked ...
0
votes
1answer
161 views

google webmaster tools, site maps and indexing, number of pages indexed reduces with no new sitemap

It seems to take two weeks ish to get updates on to the google index, which seems fair enough as I imagine googles index of the whole internet could be rather large. However, the some changes to my ...