Googlebot is the bot software that Google uses to crawl over 20 billion pages each day, the data obtained during a crawl is then analyzed and ranked on Google Search.
1
vote
1answer
204 views
Google search : pubdate issue
At some point this year, Google Search started to show the publication date.
When I updated my website, I noticed that the publication date was actually my birthday, simply because it's the first ...
1
vote
3answers
171 views
Why isnt google crawling on my blog? [duplicate]
I have a blog.I made it on August,2010,so it has been a while but I am unable to make Google crawl on my site.I have requested Google to crawl but I still there is no response.Any reasons to note why ...
3
votes
2answers
56 views
Preventing Adsense Crawling Specific Regions of the Site
In my website, I wan't adsense crawler to not consider certain parts of the HTML. Is there any snippet I can use. The reason is I am using a plugin from an external provider and as soon as I installed ...
8
votes
3answers
2k views
Does Google cache robots.txt?
I added a robots.txt file to one of my sites a week ago, which should have prevented googlebot from attempting to fetch certain URLs. However, this weekend I can see googlebot loading those exact ...
9
votes
1answer
509 views
How did Google manage to crawl my 403 pages?
I had a couple private files in a directory on my school folder. You could see that the files existed by going to myschool.edu/myusername/myfolder, but trying to access the files themselves via ...
4
votes
3answers
558 views
nofollow doesn't stop crawling?
In my website, most of the pages has a linkedin sharing button written in this way:
<li>
<a id="share_linkedin"
...
3
votes
2answers
112 views
Site re-styling affecting google ranking
I have to re-style a website in Drupal, and the client has a clear request: The site cannot drop its position on Google.
If I build a new Drupal site, with the same URLs, the same meta and title ...
2
votes
2answers
156 views
How to fix 404 crawl error of a website
I have a navigational links in a php page nav.php, Which is included in all pages of my website. I have got folder structure like below
+top_folder
-config.php
+includes_folder
...
