All web resources crawled by Google bots and recorded in Google databases.

learn more… | top users | synonyms

24
votes
6answers
2k views

Why isn't my website in Google search results?

I've made a website one month ago and it still isn't indexed by Google. I haven't change any setting in the robots.txt file. I've tried to search for site:websitename.com but I still can't see it, ...
11
votes
5answers
4k views

Does Google cache robots.txt?

I added a robots.txt file to one of my sites a week ago, which should have prevented googlebot from attempting to fetch certain URLs. However, this weekend I can see Googlebot loading those exact ...
10
votes
2answers
160 views

How to tell how old a page is?

I thought Google was more or less accurate at determining who posted a text first and who copied. However, when I use the "search tool: customized interval" the results are quite odd. I've found pages ...
8
votes
3answers
510 views

Dev site indexed by Google

I was developing a new version of our site on a subdomain (dev.oursite.com) and the robots.txt with the Disallow got replaced at some point, so the site got indexed by Google and other search engines. ...
7
votes
2answers
3k views

Google Webmaster Tools Index Status is 0 but sitemap URL shows indexed

I've added my site to Google Webmaster Tools. The site was submitted a few weeks ago. The index status shows 0 but it shows 6 URLs have been indexed in the sitemaps section. If I search in Google I ...
7
votes
1answer
318 views

home page of my site is not appear in google search, when I type site:example.com

I am totally confused, when I place site:example.com in google I see every page of my website in serp(search engine result page) except the home page. I don’t know why it’s happening but I check ...
7
votes
1answer
1k views

Should I block Wordpress archive pages from search engines?

I use WordPress and /sample-post/ URLs for my posts and /yyyy/mm/ for the archives. Google has indexed the site fully. Since the archives currently display full posts I've thought that I shouldn't ...
6
votes
3answers
240 views

How to stop Google indexing both www. and no-www versions of my website

When I do a search of my site in Google it shows both: mydomain.com www.mydomain.com In my .htaccess file I added code to redirect the non-www version to the www version, but both versions are ...
6
votes
3answers
740 views

Removing full site from Google index

I want to remove the contents from Google index. Google index include my websites huge number of pages indexed in Google index about 5,000,000 Pages earlier but now left pages are 3,025,000. I have ...
6
votes
1answer
224 views

In Google Webmaster Tools we have 3 sitemaps attributed to 1 domain

I have a website that has been on the internet for almost 10 years created in "Microsoft FrontPage" with over 900 pages. Currently in Google Webmaster tools it shows up as 2 domains and 3 sitemaps: ...
6
votes
1answer
349 views

Getting link to abstract indexed in Google Scholar

We have a large digital library with thousands of papers indexed in Google Scholar. We allow Google Scholar to index our PDFs but they're blocked unless you have a subscription. So Google has ...
5
votes
6answers
4k views

Why is Google still not indexing my !# website?

I have been working on a website which uses #! (2minutecv.com), but even after 6 weeks of the site up and running and conforming to the Google hash bang guidelines stated here, you can still see that ...
5
votes
2answers
2k views

SEO - Delete old indexed pages from Google 301 vs 404?

We're launching new website with new url structure and new content. Domain is the same. Many old pages are indexed by google. Should I use 404 or 301 and redirect to '/' for these old resources? I ...
5
votes
1answer
533 views

Robots.txt got changed to disallow crawling by accident

Our robots.txt mistakenly got changed to: User-agent: * Disallow: / This occurred about 2 weeks ago and was changed back yesterday to allow crawling. Our listed pages in the Googles have ...
5
votes
4answers
296 views

Total Indexed 0, 99 not selected in webmaster tools

I recently changed domain names from xxx.com to xxx.ca I created all the 301's submitted sitemaps and submitted a change of address to google webmaster tools on Dec 16th. I've since seen urls change ...
4
votes
5answers
4k views

How can I force Google to re-index my site?

I changed the structure of my URLs. The pages are already indexed by Google and have the following structure: /myfolder/page.apsx The new structure is: /page.aspx Now all URLs that Google knows ...
4
votes
3answers
269 views

Delay before getting URLs into Google's in web index?

4 days ago, I have submitted a sitemap.xml into Google's WebMaster Tool for a new website. It contains 3 entries. I have tested the site map and Google found not errors. In about how much time can I ...
4
votes
2answers
309 views

After URL rewriting Google still features many of the old and dirty URLs

I did a complete URL rewriting of my site some three months ago, however Google still features many of the old URLs. They're not dead links as they correctly redirect to the new URLs, but it sort of ...
4
votes
1answer
223 views

Why does Google index pages of my blog in seconds, while pages on the rest of the website take longer to get indexed

I have a questions and answers website in Portuguese about a specific subject. My Q&A website also have a blog, on a sub-directory like www.site.com/articles. I had not written on the blog for ...
4
votes
2answers
237 views

Site restyling effecting Google ranking

I have to restyle a website in Drupal, and the client has a clear request: the site cannot drop its position on Google. If I build a new Drupal site, with the same URLs, the same meta and title tags, ...
4
votes
3answers
126 views

Can Google Bot index transcript of YouTube hosted video?

We have a number of videos on YouTube, which we embed on our site via iframe/object stuff. Now we want to gain more SEO value by indexing video transcriptions. I have checked one of our video on ...
4
votes
2answers
318 views

sitemap.xml for huge online store

I'm running online store with 30,000+ products (having unique urls) which are updated every day. I want google to index my products faster. Now crawler index ~500 pages/day (reports from google ...
4
votes
2answers
1k views

Understanding Google Webmasters Tools Sitemaps Information

So, I'm an 'avid' user of google webmaster tools and analytics. I'm building an e-commerce site, all with original product descriptions and a base of 'useful' industry-relevant content. I generated ...
4
votes
1answer
160 views

How to deindex image?

I have gallery of thousands of images. All of them at some point needs to be indexed. However after some period of time, some of them should be removed from search engine index while they should ...
4
votes
1answer
236 views

Google Indexing URL that is a 301 redirect

My website URL structure is: http://www.website.com/{record-type}/{record-id}/{human-readable-name} If an invalid human readable name is provided, the page 301 redirects to the correct URL. I've ...
4
votes
3answers
920 views

nofollow doesn't stop crawling?

In my website, most of the pages has a linkedin sharing button written in this way: <li> <a id="share_linkedin" ...
4
votes
2answers
421 views

Google Webmaster Tools site removal

I had a major issue in the last week where a site was found to be showing huge numbers of pages that it shouldn't have been. Naturally Google indexed all of these pages! Rather than manually remove ...
4
votes
2answers
128 views

google search index what is fresh content

I read that sites have good reputation in google search index if it is fresh content. what does fresh content means from google's point of view? I have many pages. I have created articles and ...
4
votes
2answers
206 views

Is making AJAX site crawlable AND degrading gracefully with JS turned off possible?

According to this spec, making AJAX site crawlable by Googlebot means that you have to use hashbang (#!) links in it which means it won't degrade gracefully when JS is turned off. This might mean that ...
4
votes
2answers
719 views

Google crawling the site but refusing to index dynamic content

I am trying to get Google to index an AJAX site (davidelifestyle.com). It's crawlable with JavaScript turned off and I have also recently implemented _escaped_content_ snapshot mechanism but all ...
4
votes
1answer
109 views

Site not showing in google search, and webmaster tools showing irrelevant keywords

In Google Webmaster Tools, for my relatively new website: www.caterboxgroup.co.uk, it says there are 3 keywords: domain, registered and caterboxgroup. The two first ones do not exist in the pages ...
4
votes
2answers
632 views

cdn.domain.com indexed by Google. The content is being considered as duplicate content

I am running my website with CodeIgniter framework. I have setup cdn using cname cdn.domain.com which is being indexed by Google. And the content of the both domain.com and cdn.domain.com is having ...
3
votes
3answers
7k views

How to stop certain urls from being indexed

When I type site:example.com(using my domain obviously), I get several link errors showing up in the listing. Typically, they are of the form: /some/fixed/path/admin/unblockUser/11 I am thinking of ...
3
votes
2answers
3k views

Why the number of indexed pages decreased for a “site:” search

Recently I created a website with hundreds of thousands of unique pages. A week ago, Google had indexed about 250,000 pages. It has now decreased to 90k URLs (using a site:domain.com query). What ...
3
votes
1answer
59 views

Will Google follow link if anchor tag content is blocked?

This is a followup on a question I was asking here: Cropping images & SEO Basically, if I have something like this: <a href="<Full_Res_Image>"><img ...
3
votes
3answers
95 views

Homepage disappeared from Google results

Even that my traffic through Google raised approximately 40%, the homepage (/) of my page is gone from Google results even though I have no broken links. How can I investigate the cause?
3
votes
2answers
161 views

Do error pages get indexed?

I have just started to put custom error pages on my site. But then I started wondering if google will index these error pages. I searched this up and I didn't get anything, even searched this site, so ...
3
votes
3answers
294 views

SEO blog Indexing: wordpress.com subdomain vs a registered domain?

I've used WordPress for a few of my client's sites, mostly small businesses and eCommerce sites. I have found through Google Analytics as well as the All in One Webmaster plugin that when it comes to ...
3
votes
3answers
589 views

Google is indexing unintended content that is either unpublished or a secret part of site

Google is indexing parts of my site that I haven't linked to, it's a mobile version of website that I'm working on - www.domain.com/m/ <-- note the M. How is that even possible? Only thing I can ...
3
votes
2answers
36 views

How to remove a page from site without affecting google serp

I have a travel website. Just for information purposes, I had put a weather page. Now I realize that this page is increasing my overall bounce rate; because people who are looking for the weather ...
3
votes
1answer
841 views

What do I put in htaccess to create a 410 error page when a URL parameter is present?

I tried to create a 410 error page but realized that I actually created a 302 redirect to a 410.php page. Instead I want to use a 410 error to block out an infinite number of dynamically created pages ...
3
votes
2answers
100 views

Should I replace non-user friendly URLs with friendly ones, and if so, how?

I have just added my site to Google Web Master Tools and discovered that some links (for example: In the section "Search Query / Top pages") have a URL format like: example.com/page.php?id=1234 This ...
3
votes
4answers
3k views

How can I get a list of all indexed pages for my domain?

We're in the process of migrating an outdated e-commerce cart to Magento. Part of this process will entail configuring proper 301 redirects. I'm aiming to get a list of all indexed pages to throw ...
3
votes
4answers
255 views

How does Google have more results than I have pages?

Our site has about 6500 pages. These are kept up to date in our sitemap.xml. When I go to our Google Webmaster Tools account I see that Google has indexed about 6000 of our pages. This seems quite ...
3
votes
3answers
2k views

Google webmaster Index Status. Total Indexed=0

I previously changed my domain from www.visualstudiolearn.blogspot.com to www.visualstudiolearn.com. I had around 300 posts with the previous domain name and most of them where showing up on Google. ...
3
votes
3answers
45 views

Indexed webpages stopped rising, while Submited keep rising normally

Last months I have noticed a strange behaviour in Google's Webmaster Tools. Although I provide a valid Sitemap for my website, which is normally processed and without any issues from Google, while ...
3
votes
1answer
73 views

Google Search for my domain has about 3500 3-rd level subdomain I did not create

Some time ago I created the domain in .com zone. Now I tried to evaluate the number of pages are in Google index using request like site:mydomain.com but surprisingly there were a lot of results like: ...
3
votes
1answer
103 views

How can I force a certain URL for Google to use in search results?

Each page on my site can be accessed by a shortened URL and normal "long" URLs. The shortened URL redirects to the long URL and the long URL includes an optional title that's not parsed (much like on ...
3
votes
1answer
959 views

Google Webmaster Tools Index Status 0 for one year

I've read this topic with answers but they do not address why zero for one year? but other areas in Google Webmaster Tools report 400 URL indexed. Google Webmaster Tools Index Status Only 2 Indexed. ...
3
votes
1answer
401 views

Does submitting 'URL and all linked pages' help with new website indexing?

I am aware that sitemaps are one way to provide lists of page URLs to Google bots. However, Google webmaster allows one to fetch a page as Google and to submit the 'URL and all linked pages'. Does ...