All web resources crawled by Google bots and recorded in Google databases.

learn more… | top users | synonyms

30
votes
5answers
2k views

Why isn't my website in Google search results?

Its been sometime and Google has yet to index my site, how can I get Google to index my website?
14
votes
7answers
5k views

Does Google cache robots.txt?

I added a robots.txt file to one of my sites a week ago, which should have prevented googlebot from attempting to fetch certain URLs. However, this weekend I can see Googlebot loading those exact ...
13
votes
2answers
396 views

How to tell how old a page is?

I thought Google was more or less accurate at determining who posted a text first and who copied. However, when I use the "search tool: customized interval" the results are quite odd. I've found pages ...
12
votes
1answer
635 views

Why are these sites cloning my content, and how do I stop them?

So I was searching for one of my own websites on Google: http://www.botlibre.com and to my surprise found some clones of my website, like: http://83waystowasteyourtime.adidas.com and: http:// ...
10
votes
2answers
7k views

Google Webmaster Tools Index Status is 0 but sitemap URL shows indexed

I've added my site to Google Webmaster Tools. The site was submitted a few weeks ago. The index status shows 0 but it shows 6 URLs have been indexed in the sitemaps section. If I search in Google I ...
9
votes
3answers
2k views

How do I find when an URL was first indexed by Google?

How do I find out when a particular URL was first indexed by Google? I'd prefer a solution that works even for competitors' URLs that are not owned by me.
8
votes
3answers
660 views

Dev site indexed by Google

I was developing a new version of our site on a subdomain (dev.oursite.com) and the robots.txt with the Disallow got replaced at some point, so the site got indexed by Google and other search engines. ...
7
votes
3answers
1k views

Removing full site from Google index

I want to remove the contents from Google index. Google index include my websites huge number of pages indexed in Google index about 5,000,000 Pages earlier but now left pages are 3,025,000. I have ...
7
votes
1answer
359 views

home page of my site is not appear in google search, when I type site:example.com

I am totally confused, when I place site:example.com in google I see every page of my website in serp(search engine result page) except the home page. I don’t know why it’s happening but I check ...
7
votes
1answer
2k views

Should I block Wordpress archive pages from search engines?

I use WordPress and /sample-post/ URLs for my posts and /yyyy/mm/ for the archives. Google has indexed the site fully. Since the archives currently display full posts I've thought that I shouldn't ...
6
votes
6answers
5k views

Why is Google still not indexing my !# website?

I have been working on a website which uses #! (2minutecv.com), but even after 6 weeks of the site up and running and conforming to the Google hash bang guidelines stated here, you can still see that ...
6
votes
3answers
687 views

Number of indexed pages with “site:” search less than reported in Google Webmaster Tools

In Google webmaster tools it shows that I have 309 pages indexed, but when I search using "site:site.com" appear about 180 only (On the first page appears "about 608 results," something near the total ...
6
votes
3answers
417 views

How to stop Google indexing both www. and no-www versions of my website

When I do a search of my site in Google it shows both: mydomain.com www.mydomain.com In my .htaccess file I added code to redirect the non-www version to the www version, but both versions are ...
6
votes
1answer
275 views

In Google Webmaster Tools we have 3 sitemaps attributed to 1 domain

I have a website that has been on the internet for almost 10 years created in "Microsoft FrontPage" with over 900 pages. Currently in Google Webmaster tools it shows up as 2 domains and 3 sitemaps: ...
6
votes
1answer
461 views

Getting link to abstract indexed in Google Scholar

We have a large digital library with thousands of papers indexed in Google Scholar. We allow Google Scholar to index our PDFs but they're blocked unless you have a subscription. So Google has ...
5
votes
2answers
2k views

SEO - Delete old indexed pages from Google 301 vs 404?

We're launching new website with new url structure and new content. Domain is the same. Many old pages are indexed by google. Should I use 404 or 301 and redirect to '/' for these old resources? I ...
5
votes
1answer
559 views

Robots.txt got changed to disallow crawling by accident

Our robots.txt mistakenly got changed to: User-agent: * Disallow: / This occurred about 2 weeks ago and was changed back yesterday to allow crawling. Our listed pages in the Googles have ...
5
votes
2answers
471 views

sitemap.xml for huge online store

I'm running online store with 30,000+ products (having unique urls) which are updated every day. I want google to index my products faster. Now crawler index ~500 pages/day (reports from google ...
5
votes
4answers
324 views

Total Indexed 0, 99 not selected in webmaster tools

I recently changed domain names from xxx.com to xxx.ca I created all the 301's submitted sitemaps and submitted a change of address to google webmaster tools on Dec 16th. I've since seen urls change ...
4
votes
3answers
9k views

How to stop certain urls from being indexed

When I type site:example.com(using my domain obviously), I get several link errors showing up in the listing. Typically, they are of the form: /some/fixed/path/admin/unblockUser/11 I am thinking of ...
4
votes
5answers
6k views

How can I force Google to re-index my site?

I changed the structure of my URLs. The pages are already indexed by Google and have the following structure: /myfolder/page.apsx The new structure is: /page.aspx Now all URLs that Google knows ...
4
votes
4answers
206 views

Huge drop off in Google “indexed” pages after switching to HTTPS, should I switch back to HTTP?

I did a full redesign of my website to make it responsive (bootstrap), added localization, and also switched the whole website to use HTTPS. My website use to default to http://www.domain.com/ and ...
4
votes
3answers
338 views

Delay before getting URLs into Google's in web index?

4 days ago, I have submitted a sitemap.xml into Google's WebMaster Tool for a new website. It contains 3 entries. I have tested the site map and Google found not errors. In about how much time can I ...
4
votes
2answers
472 views

Is it bad for SEO to submit a partially finished site to Google?

Hypothetical Situation 1: I have a plan for an amazing large-scale website with lot's of really cool features, animations and interactivity. This will take a several months to completely develop. ...
4
votes
1answer
274 views

Why does Google index pages of my blog in seconds, while pages on the rest of the website take longer to get indexed

I have a questions and answers website in Portuguese about a specific subject. My Q&A website also have a blog, on a sub-directory like www.site.com/articles. I had not written on the blog for ...
4
votes
2answers
4k views

Why the number of indexed pages decreased for a “site:” search

Recently I created a website with hundreds of thousands of unique pages. A week ago, Google had indexed about 250,000 pages. It has now decreased to 90k URLs (using a site:domain.com query). What ...
4
votes
2answers
316 views

After URL rewriting Google still features many of the old and dirty URLs

I did a complete URL rewriting of my site some three months ago, however Google still features many of the old URLs. They're not dead links as they correctly redirect to the new URLs, but it sort of ...
4
votes
1answer
3k views

What do I put in htaccess to create a 410 error page when a URL parameter is present?

I tried to create a 410 error page but realized that I actually created a 302 redirect to a 410.php page. Instead I want to use a 410 error to block out an infinite number of dynamically created pages ...
4
votes
6answers
5k views

How can I get a list of all indexed pages for my domain?

We're in the process of migrating an outdated e-commerce cart to Magento. Part of this process will entail configuring proper 301 redirects. I'm aiming to get a list of all indexed pages to throw ...
4
votes
2answers
630 views

Informing Google of URL Structure Change

I have a website and the URLS were constructed like so: /index.php/Topic /index.php/AnotherTopic These were indexed in google and search results returned that pointed to these. However, I've ...
4
votes
2answers
279 views

Site restyling effecting Google ranking

I have to restyle a website in Drupal, and the client has a clear request: the site cannot drop its position on Google. If I build a new Drupal site, with the same URLs, the same meta and title tags, ...
4
votes
3answers
408 views

Can Google Bot index transcript of YouTube hosted video?

We have a number of videos on YouTube, which we embed on our site via iframe/object stuff. Now we want to gain more SEO value by indexing video transcriptions. I have checked one of our video on ...
4
votes
3answers
1k views

nofollow doesn't stop crawling?

In my website, most of the pages has a linkedin sharing button written in this way: <li> <a id="share_linkedin" ...
4
votes
2answers
1k views

Understanding Google Webmasters Tools Sitemaps Information

So, I'm an 'avid' user of google webmaster tools and analytics. I'm building an e-commerce site, all with original product descriptions and a base of 'useful' industry-relevant content. I generated ...
4
votes
1answer
290 views

How to deindex image?

I have gallery of thousands of images. All of them at some point needs to be indexed. However after some period of time, some of them should be removed from search engine index while they should ...
4
votes
2answers
701 views

Microdata for product images which aren't displayed in the page?

My internet shop displays thumbnails of products' images, and I see that these thumbnail are (slowly) being indexed by Google. I want also full images to be indexed. Can I use Microdata for this? ...
4
votes
1answer
267 views

Google Indexing URL that is a 301 redirect

My website URL structure is: http://www.website.com/{record-type}/{record-id}/{human-readable-name} If an invalid human readable name is provided, the page 301 redirects to the correct URL. I've ...
4
votes
1answer
55 views

Why doesn't Google index the translated versions of my page?

I have a website in Persian language and translated some of my popular posts into English and added this header in <head> section in both of the pages (not entire site): <link ...
4
votes
1answer
332 views

getting CDN images indexed with Google

I have somewhere close to 500,000 user-uploaded images hosted on a Cloudfront CDN -- separate from our main host (exampledomain.com). Up until this point, few of them have been getting indexed at the ...
4
votes
2answers
616 views

Google Webmaster Tools site removal

I had a major issue in the last week where a site was found to be showing huge numbers of pages that it shouldn't have been. Naturally Google indexed all of these pages! Rather than manually remove ...
4
votes
2answers
155 views

google search index what is fresh content

I read that sites have good reputation in google search index if it is fresh content. what does fresh content means from google's point of view? I have many pages. I have created articles and ...
4
votes
2answers
221 views

Is making AJAX site crawlable AND degrading gracefully with JS turned off possible?

According to this spec, making AJAX site crawlable by Googlebot means that you have to use hashbang (#!) links in it which means it won't degrade gracefully when JS is turned off. This might mean that ...
4
votes
2answers
844 views

Google crawling the site but refusing to index dynamic content

I am trying to get Google to index an AJAX site (davidelifestyle.com). It's crawlable with JavaScript turned off and I have also recently implemented _escaped_content_ snapshot mechanism but all ...
4
votes
1answer
77 views

Google Cache: Blank Page (Responsive Design) [closed]

I've been aware of this issue for sometime but sadly not had the downtime to address, but I want to ensure that its actually a problem before wasting my time. The problem that I have is that Google's ...
4
votes
1answer
272 views

Our company's white label site was removed from Google's search index, would canonicalization help?

I recently started working for a new company and they have a lot of white label sites. Just last week I checked Google Webmaster Tools for one of our bigger in-house white label sites (one we ...
4
votes
1answer
118 views

Site not showing in google search, and webmaster tools showing irrelevant keywords

In Google Webmaster Tools, for my relatively new website: www.caterboxgroup.co.uk, it says there are 3 keywords: domain, registered and caterboxgroup. The two first ones do not exist in the pages ...
4
votes
2answers
949 views

cdn.domain.com indexed by Google. The content is being considered as duplicate content

I am running my website with CodeIgniter framework. I have setup cdn using cname cdn.domain.com which is being indexed by Google. And the content of the both domain.com and cdn.domain.com is having ...
4
votes
0answers
55 views

Index status shows significantly less pages than search results? *Not a New Site

I am trying to understand how within Google Webmaster Tools (Google Index>Index Status) I can have a figure of 2,000 total indexed pages (and falling daily) yet when I search Google using the (site:) ...
4
votes
2answers
52 views

Possible to check which links in a sitemap are indexed and which ones aren't?

Is it possible to check which links of the sitemap I've submitted are being indexed, and which ones are not? I've submitted my sitemap.xml without errors. When I check the sitemap in google search ...
3
votes
3answers
680 views

How to make Google index millions of user profile pages?

I have a website with a lot of user profiles (like LinkedIn or the like). I need Google to index each public user profile page. I have a few things on my mind: 1- There is no user listing on my ...