6

I am working on a project to create a URL shortener. I have successfully gotten it to create short URLs, but I would like to submit them to Google. When I submit a short URL via Google Webmaster Tools, it gives me a redirect error.

I see shortened URLs for goo.gl and bit.ly in the search results:

Example of site:goo.gl search

How do they do that? How are they able to submit their short URLs to Google? Are they using curl? When Google fetches the URL do they identify it as Googlebot, download the contents with curl and re-serve the actual contents to Google? Could I do something similar? I am using PHP.

Having used the internet for ten years I had never before seen any search result linked with a shortened URL in Google! Why am I only seeing it now? From my research, I suspect:

  • Due to duplicate content Google does not want to show fake websites on top of the search results.
  • If a user clicks on a Google search result, and after click, that link redirects to another URL, Google will demote the redirecting website.

My aim is to create a short URL website and submit all short URLs to Google. With many URLs submitted, there would be many chances to rank in different Google searches.

migrated from wordpress.stackexchange.com Dec 23 '15 at 17:54

This question came from our site for WordPress developers and administrators.

  • Are your short URLs redirecting to URLs on other sites ("off-domain") as in a generalised URL shortening service OR to URLs on the same site ("on-domain")? From your wording ("fake websites", "demote the redirecting website", "a short URL website") I assume the former? But that would also raise issues of legality and something that Google would actively try to avoid? – MrWhite Dec 27 '15 at 1:57
4

Just because a URL is in Google's index doesn't mean that it was submitted through webmaster tools or search console. Google includes most URLs in the index only because it finds links to them on other sites.

Most of the time Google won't index a redirecting URL (including a shortened URL.) Usually Google prefers the destination. If the redirect URL has a lot of links and the destination is not crawlable Google may choose to index the redirect instead. If the redirect URL is more authoritative (like a home page URL that redirects to a deep page), Google may also choose to index the shortened URL.

Site search results are also very different than other types of search results. Google is much more likely to show poor quality pages in a site search result than in normal search. Part of why you are finding some redirect URLs for goo.gl is because you are using site search. Those URLs are very unlikely to show up for normal search phrases.

If you create a URL shortener, you won't be able to get many of your shortened URLs indexed by Google.

  • 2
    "Site search results are also very different" - yes, I think this is exactly why the shortened URLs are showing at all in these search results. If you do a "normal" exact match search (ie. surround the search phrase in quotes) for any of the titles shown in the screenshot then the shortened goo.gl URL is not returned at all. Only by explicitly including site:goo.gl in the search can you get these shortened URLs returned in the SERPs. – MrWhite Dec 23 '15 at 19:11
2

To build on Stephen Ostermiller's answer, there are (at least) two things you can try to persuade Google to index your short URLs:

  1. Use 302 redirects instead of 301. A 302 redirect is "temporary", and hints to search engines that the original, unredirected URL is the stable and authoritative one. In particular, while the details of Google's ranking algorithms are obviously not public, it's generally accepted that 302 redirects don't pass "PageRank" to the target page, at least not to the same extent as 301 redirects do.

    Obviously, there's only so much you can do this way — you can't just "steal" another page's ranking simply by making a 302 redirect to it — but if your short URL is the most commonly linked URL for the page, and you're using a 302 redirect instead of 301, it's likely that Google may indeed show the short URL in search results.

  2. Add a rel=canonical link to the target page head section, linking back to your short URL. This is a much stronger endorsement; it's basically the target page telling Google "Don't show this URL in search results, but show that short URL instead."

    This should pretty much guarantee that your short URL will show up in search results (at least assuming that you also use a 302 redirect — having a canonical link tag pointing to a 301 redirect might give weird or unexpected results, since the link and the redirect will basically contradict each other). Of course, this method is only possible if you can actually control the target page.

  • If my short links will not appear in search result, so it is useless for me,, i want to appear my short list on normal user search... – Ypages Onine Dec 26 '15 at 17:22
  • 1
    If these are off-domain redirects (which I suspect) then a 302 (as opposed to a 301) redirect is unlikely to benefit in this respect. As Matt Cutts said on his blog back in 2006, with regards to 302s and off-domain redirects: "Google is moving to a set of heuristics that return the destination page more than 99% of the time." - and a 3rd party URL shortening service is not going to satisfy the 1% of edge cases. And if this is a 3rd party URL shortening service they aren't going to be able to add a canonical link element. – MrWhite Dec 31 '15 at 16:51
  • I know this is an old question, but to anyone thinking about implementing a site like this: if I, as a webmaster, saw 302s instead of 301s on my short URLs, I'd instantly stop using that service. – Ivo van der Veeken May 27 '17 at 14:53

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.