19 votes
Accepted

How do I find when an URL was first indexed by Google?

To know the age of an URL you can follow this link by replacing www.example.com by the URL you want: https://www.google.com/search?tbs=cdr%3A1%2Ccd_min%3A1%2F1%2F2000&q=site%3Ahttp%3A%2F%2Fwww....
user avatar
  • 10k
17 votes

Why are these sites cloning my content, and how do I stop them?

Okay. This is a rather odd one, but not as tragic as you would think. I am not sure what the payoff would be. This makes absolutely no sense to me as to why someone would do this. I cannot see an ...
user avatar
  • 32.5k
14 votes

What can cause "Discovered - currently not indexed" in the new GWT

It is just part of the process. There is nothing for you to do until you actually get errors. To understand the categories, you need to understand how indexing works which is something that is done ...
user avatar
  • 5,867
12 votes

How to tell how old a page is?

I researched the answer to this question this way: using Google since this is the example I have, how Google gets creation dates and modified dates, and date formats that Google recognizes. Please ...
user avatar
  • 32.5k
8 votes
Accepted

Remove subdomains from Google index and stop indexing them

You can remove the sub-domains in webmaster tools, but first you need to add the sub domains as seperate sites and then submit a site removal. They should be gone within a day or so. See these ...
user avatar
  • 2,466
8 votes

How do I find when an URL was first indexed by Google?

Zistoloen found a way to have Google display the date when it first indexed the content of the page. I'm adding it to my answer as well because I think I can explain it more clearly. Search Google ...
user avatar
8 votes

How to make Google index millions of user profile pages?

Google is only going to index millions of pages from a site with lots of reputation. I'd estimate that your site would have to have at least PageRank 6 before Google were willing to index that many ...
user avatar
7 votes
Accepted

Number of indexed pages with "site:" search less than reported in Google Webmaster Tools

You do not have an indexing problem. Google Webmasters Tools is the authoritative source of information about your website with Google. Whatever it says there is true. Operators like site: and link: ...
user avatar
  • 85.7k
7 votes
Accepted

Is it bad for SEO to submit a partially finished site to Google?

There is no problem with showing Google a small site that you plan to add to later. If you have one page of good content, Google would like to index it because it could be useful to somebody ...
user avatar
7 votes
Accepted

Does Google index #! (hash bang) URLs?

Hash bang URLs are most definitely deprecated. Wix (a popular web site builder) is probably one of the biggest facilitators of getting people stuck in them. The problem comes into play when you can ...
user avatar
7 votes
Accepted

Ampersand (&) in actual URL and sitemap

if this difference of ampersand in URL and sitemap will cause any issue. tl;dr No issue, because the URLs are the same. Since in sitemap & has to be escaped I replaced & with & ... ...
user avatar
  • 40.9k
6 votes
Accepted

Robots.txt got changed to disallow crawling by accident

If Google has started adding pages again, then you are fine. We had this happen with a few sites that were pushed from development into production and their robots.txt was not changed. In fact, ...
user avatar
6 votes

Number of indexed pages with "site:" search less than reported in Google Webmaster Tools

You had posted this question differently twice. This answer was to the question marked as duplicate which I now post here hoping to help you. While the two questions may not be exactly the same, this ...
user avatar
  • 32.5k
6 votes

Huge drop off in Google "indexed" pages after switching to HTTPS, should I switch back to HTTP?

We just switched from HTTP to HTTPS two weeks ago, and after doing extensive research this is what we did: Added the HTTPS property to our Search console. Redirected HTTP robots.txt to HTTPS robots....
user avatar
  • 639
6 votes

Remove HTTPS site from Google search index because users are getting a certificate error

I'm very surprised that Google is indexing the HTTPS pages with a broken certificate. Google usually prefers the HTTPS site if they both work, but I've never seen it prefer HTTPS when there is a ...
user avatar
5 votes

Number of indexed pages with "site:" search less than reported in Google Webmaster Tools

It could also be duplicate content filters. Do a site search then go to last page of search results. This may be page 30 or more. At the bottom of the last page you should see something similar to ...
user avatar
  • 3,319
5 votes

Where are the complete set of Highwire Press metatags defined?

Note: the link to this question has come up often in searching. Thought it deserved an attempt at an answer. I've found three resources thus far, but everything seems to be woefully incomplete. ...
user avatar
  • 151
5 votes

can google index/crawl random subdomains that aren't linked to from anywhere

Yes, Google can do that and it is best to assume that anything that is publicly available on the internet may be indexed by Google. Linked to or not. Of course, if you don't link to it the chances of ...
user avatar
  • 4,497
5 votes
Accepted

Prevent Google from seeing "svg" as a content keyword without using .htaccess

Your problem is not that SVG images are getting indexed. The items in your screenshot are image directories. If you visit those URLs you will likely see a list of files that are in the directories. ...
user avatar
5 votes

Is it possible for Google to Index a site with login?

This sounds like a great use-case for First Click Free - which would allow you to show the full content to Googlebot, while limiting the number of times a user can access content from there.
user avatar
  • 7,303
5 votes
Accepted

Is there a penalty for submitting URLs for indexing too often?

Unless you are effectively spamming Google with 1000's of submissions then there would be no reason to receive some kind of penalty that would effect your own site submissions. However, for your own ...
user avatar
  • 4,267
5 votes
Accepted

How can I remove this URL from Google?

It looks like that you have affiliate links which have 302 redirected to the Amazon or another affiliate site. This is a common issue for affiliate websites. The issue with 302 redirects is that ...
user avatar
  • 544
4 votes
Accepted

Does noindex metatag have a negative effect on the canonical URLs PageRank?

You should not use both noindex and rel=canonical, because noindex won't let pages pass any PageRank to its canonical version. rel="canonical" is a hint, not a directive, so Google will decide ...
user avatar
4 votes

How do I find when an URL was first indexed by Google?

There may not be any way to find out when an arbitrary web page was first indexed by Google — certainly I don't know of any way to do so. It's possible that Google simply does not store that ...
user avatar
4 votes
Accepted

Two kinds of Google index

Google looks at things differently than we do. Sorry. That is just the way it is. Sometimes you have to see things from Google's perspective before the data makes sense. The Google Webmaster Tools ...
user avatar
  • 32.5k
4 votes

Huge drop off in Google "indexed" pages after switching to HTTPS, should I switch back to HTTP?

Are you checking the correct version of the search console? See the Google support site point 4: you need to add the https version as a new site. If you are still looking at the http version it's ...
user avatar
4 votes
Accepted

Changing URLs in Google's Index

First things first. Of your wrong URLs, make sure you are using a 301 redirect to the right URL and not just rewriting them. Make sure your links are correct. Update your sitemap so that it is ...
user avatar
  • 32.5k
4 votes

301 and no-index tag don't work

I'm going to have to say google is too slow. Your best bet is to go into Google search console, access your domain within it, then select the gear icon and then site settings, then for crawl rate, ...
user avatar
4 votes
Accepted

How to remove old style URLs from the Google Index completely and replace them with a new style?

If you would like to replace: https://example.com/username/branches with https://example.com/username/andOtherUrlsAppendedtothis Then all you need to do is set a permanent 301 redirect. Considering ...
user avatar
  • 3,143
4 votes

How long before Google indexes a new (to me) domain that previously had spam and virus problems?

This is the very interesting situation as you didn't know the history of the domain. There are several things you may need to look. Virus vs Spam: Check if it had infection issue or it was penalised ...
user avatar
  • 3,143

Only top scored, non community-wiki answers of a minimum length are eligible