14

I truly believe that any webmaster entering any niche should investigate their online competition beforehand to know what they're up against and to see if their goals are realistic and affordable in both time and money.

As you may know, there are a few ways of investigating your competition online. Rankings, local rankings, citations, social mentions and of course backlinks. This question is not related to any of the above but is related somewhat because it has the same end goal which you'll see momentarily.

So hopefully my question and the answers to follow may aid some other webmaster encountering the same problem. I'll illustrate the problem as a scenario that'll hopefully give you an idea of what I believe and what I'm asking.

The Scenario

Your website has been ranking top for almost 2 years within a fairly competitive niche. Then, out of nowhere, you drop in ranking and a site that has never been on page 10, never mind position 10, is all of a sudden in top.

An immediate reaction might be foul play and you investigate their content and backlinks, only to find that their content is extremely thin and they have zero backlinks. So the possibility here is that the backlinks have not indexed for that site with whatever tool you use to check for these yet. Or, Google for whatever reason has made an error that will resolve itself.

4 months on and the same thin site with 0 backlinks (Only those from expected bot/scrapers - webstats etc) is still top in rankings.

So it looks like this site is here to stay and you start thinking about other reasons to why your site is being outranked.

One possible reason could be that your own rankings has slipped, along with many other sites but having all those sites drop in ranking is unrealistic so the only way forward would be to believe that they have earned their rankings through either reputation with Google or foul play. The conclusion here would be - or least mine - that the site in question has recently moved from one domain to another. They may just have a simple 301 re-direct from one domain to another, or they may have told Google within Google Webmaster Tools they have moved site. Scenario over.

The Question

My question is fairly simple and I apologise for the long paragraphs of text but I want a good idea of what is trying to be achieved here. Enough typing, here's the question!

Is it possible to find out if one domain has moved from one domain to another, either using 301 re-directs or moving in Google Webmaster Tools?

Improve this question
10
  • I would have thought the same tools that track backlinks could potentially track this information also. However, I have my doubt whether (many of) these tools actually follow links and therefore don't see the redirection (although that would also mean their backlink reporting wouldn't be entirely accurate).
    – MrWhite
    Aug 5 '13 at 20:45
  • +1. Great question. I assumed that when you move a site from within Webmaster Tools, that Google automatically adjusted the results from link:example.com in order to 'move the link juice', so to speak. This scenario is weird, to be quite frank.
    – Athoxx
    Aug 22 '13 at 9:47
  • I have an idea, could you share URL with no backlinks?
    – Zistoloen
    Aug 29 '13 at 10:47
  • 2
    @Zistoloen, then it would have a backlink, heh
    – Octopus
    Sep 11 '13 at 19:20
  • @Octopus: a nofollow backlink.
    – Zistoloen
    Sep 11 '13 at 20:53
3
+50

Knowing the 301 redirected URL is possible in several cases but not all the time.

Several online tools like Ahrefs or MajesticSEO can display this kind of information for you. You just need to look at backlinks classified under "Redirect" status.

Example for Ahrefs:

enter image description here

However, you need to know a good SEO professional don't want people find the 301 redirect because in most of the cases, the redirected URL received in the past very good backlinks from good SEO places (websites with high PageRank for example).
That's why in general, in order to hide these good SEO places (and thus maybe good opportunities for everyone to get very good backlinks), you can't find the redirected URL with online tools. Indeed, the professional has blocked for the redirected URL all robots (especially those from Ahrefs, MajesticSEO...) except those from Google.
As a consequence, the 301 redirect is powerful for SEO and hidden for people.

P.S.: cloaking can also be applied by the professional to hide backlinks to the redirected URL for visitors. Therefore, these good places for SEO are even better hidden.

Improve this answer
5
  • +1 As I can see this as a semi-working method. Depends if the redirect is detected. The problem is doesn't ahrefs and majestic use Google index? to discover? a redirecting domain won't be indexed, And I assume the ones shown are for sites with internal.com/?redirect=go:domain or something like that.
    – Simon Hayter
    Sep 14 '13 at 12:42
  • 1
    Indeed, this is a semi-working method but I believe this is the best method we can find.
    – Zistoloen
    Sep 14 '13 at 14:01
  • Shame Google don't do a cache going back months to years ;\
    – Simon Hayter
    Sep 14 '13 at 14:09
  • Yes, Google is not perfect...
    – Zistoloen
    Sep 14 '13 at 14:23
  • 1
    I don't know about ahrefs but I am pretty sure majesticSEO has their own crawler and doesn't rely on Google for their index.
    – Joshak
    Oct 23 '13 at 16:44
2

There are several sites that will tell you all sorts of information about sites such as:

  • Number of pages on the site
  • Traffic estimates for the site
  • Other sites hosted at the same IP address (almost what you want)
  • Other sites registered to the same owner (again, almost what you want)

I could not find one that tells you what other domain names redirect to a site. It probably would not be a huge stretch for one of those sites to add this type of functionality.

If I were to try build a tool to answer this question, here is what I would do:

  • Download the entire list of domain names that have DNS entries. This in itself is a project. You have to apply to each registry for access or pay a broker that has done that legwork. This question on stackoverflow has answers for both approaches
  • Download a page from each of the 150 million domain names to see if it redirects to some other site. I figure you should be able to do about 50 per second and the crawl would take a little over a month. This task is shardable, so it could be sped up by having multiple machines working on it. The bandwidth and hosting costs for this crawler could end up being sizable, the bandwidth for 50 sites per second would probably be about 20mb/s assuming an average download of 50KB pre site.
Improve this answer
3
  • Nice try Stephen but you don't really give the answer.
    – Zistoloen
    Sep 12 '13 at 16:02
  • I'm hoping something better will come up, but after doing some research, I doubt it.
    – Stephen Ostermiller ♦
    Sep 12 '13 at 16:06
  • +1 for the DNS look up, through I had already done that, I checked who else was using the same name servers as well as the IP. Through I suspect the name servers are not the same and the IP is completely different.
    – Simon Hayter
    Sep 12 '13 at 17:07
1

I'd suspect this is more effective using the cross-domain canonical tag. Using the canonical tag would keep the non-indexed site active so you could continue building links that pass on to the money site (It's hard to send a link request for a page that has already been 301'd).

There aren't any automated/off the shelf ways that I'm aware of to detect 301s. And I don't believe there are any ways to be confident you found them all. A few options that might help you find some are:

DNS Lookup to see if you can find other domains by the same owner. This will give you a good starting place if you find one. There are some websites that are used to try and find networks of sites (spyonweb.com I believe is one) they use AdSense ID, affiliate ID, DNS info etc. In this case I wouldn't think the owner would leave any of these footprints but you never know.

Open source web crawler, should be able to be tweaked to only follow links that 301 and return results for your desired domain name.

Look for link sellers in your niche. This is probably being done for the sake of creating a buffer between the money site and possible bad links (it's much easier to shut off a single redirected site then it is to get a few hundred webmasters to remove links to your site if you get penalized). Could mean they are participating in link buys, using something like Xenu on the sellers site would turn up any 301s.

inanchor: search on Google. If there is an identifying brand that it ranks well for you might turn up some links that use the brand in the anchor text but go to a different domain (which then hopefully redirects to the domain you want).

If you happen to find 1 or 2 sites using these methods you'll likely turn up a pattern that can be used to find more.

All these would be moot however, if they are using the canonical tag instead of 301. I'm sure there are some additional creative searches you could do but these should give you a good start.

-Good luck

Improve this answer
1
  • Digging through some reports in MajesticSEO it looks like their site explorer tool has redirects split out in their backlinks section. May give you what you need.
    – Joshak
    Sep 25 '13 at 13:57
-1

If you want to know what 301 redirects are in place currently, yes that is absolutley possible with tracert on the command line. It should show you each hop, you may have to look up a couple of IP addresses, but it will show you the 301 redirects when the URL is requested from the server.

I don't know if there is a tool that would record the 301 itself. You might have some luck with Wayback Machine, but I would not be too hopeful.

Historical SERPs and backlink checkers should have all the relevant domains that would be redirected.

Here is a web version I found, used one of my bitly links to show how it works. Tracks two 301s to my domain. enter image description here https://wheregoes.com/trace/2021789857/

Improve this answer
3
  • 1
    I think this is the reverse of what they were asking for. They want to get a list of all the URLs that redirect TO some URL. This tool gives you the redirect chain FROM some URL.
    – Stephen Ostermiller ♦
    Mar 3 '21 at 17:55
  • I don't think there is a way to get all the URLs directed at a domain using 301 without seeing htaccess or other server config. I did mention that you should be able to find them using backlink tools and historical SERPs, there isn't a less brute force way that I know of. But, I find it highly unlikely that some unknown domain has that much power without appearing in a historical SERP for that term. That is just my opinion, but I would bet on it being there.
    – keepkalm
    Mar 3 '21 at 18:33
  • Think about it this way, how many urls could there be from one domain with 301 redirects? It is an impossible task to find them all.
    – keepkalm
    Mar 27 '21 at 15:54

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.