Take the 2-minute tour ×
Webmasters Stack Exchange is a question and answer site for pro webmasters. It's 100% free, no registration required.

A few months ago I received the following notification on Google Webmaster for a website I look after.

Unnatural links to your site—impacts links Google has detected a pattern of unnatural artificial, deceptive, or manipulative links pointing to pages on this site. Some links may be outside of the webmaster’s control, so for this incident we are taking targeted action on the unnatural links instead of on the site’s ranking as a whole. Learn more.

The question here is, should we actively attempt to disavow these links given that the action is seemingly targeted to just a bunch of keywords? I've downloaded the inbound links sample from Google Webmaster and so far I've been through the disavow and reconsideration requests process 6 times, each taking 2-3 weeks only to be supplied just 2 more links that Google don't approve of. At this rate it will take me the rest of my natural life to cleanup all these spammy links!

It seems disavowing is futile as they haven't implemented broad actions against the website as a whole and (from what I can gather) have already nullified the value of those offending links. Under the quoted statement above however is a reconsideration request button that seems to imply I should be actively doing something here?

UPDATE 14th October --

I have since created a small .NET application that you can feed the CSV sample links file into from Google Webmaster. What this tool does is crawl all the links and looks for specific linking patterns as per some configurable match strings.

I realised that many of the links that Google are taking issue with were created by a rogue SEO firm we hired several years ago. All the links are appended with 1 of 5 different descriptions. The application I built uses some regexes to isolate any link sources with these matching appendages and automatically builds the disavow txt file.

In the end it had to come down to an algorithm as manually disavowing links on this scale would take weeks!

I will post the app here once I've cleaned it up.

share|improve this question
    
Waiting for the App.. Good work –  user2434 Oct 14 '13 at 10:20
    
OK, it needed a bit of work but you'll find a link to the tool in my answer below. –  QF_Developer Oct 14 '13 at 17:13
add comment

marked as duplicate by Stephen Ostermiller Nov 6 '13 at 16:35

This question has been asked before and already has an answer. If those answers do not fully address your question, please ask a new question.

2 Answers

Google has said that the unnatural links message only gives you a handful example links, not all the links that need to be removed.

Even the list of links in Google Webmaster Tools is not complete.

Here is an article about a better disavow proceedure: http://www.searchenginejournal.com/how-to-know-which-links-to-disavow-in-google/50709/ It suggests using multiple sources to find a more complete list of links to your site:

  • Google Webmaster Tools
  • Majestic SEO
  • SEOMoz‘s Open Site Explorer
  • Ahrefs

It also suggests how to do the filtering in Excel spreadsheets that would take the place of the .net program you have written.

share|improve this answer
    
Looks very comprehensive, I'll refer to this guide if the latest attempt fails. From what I've read the list of links in Google are pretty comprehensive and in each case Google have turned up the same link patterns, which are not difficult to isolate. I'm not sure I need anything more sophisticated than the tool I built, although time will tell. –  QF_Developer Oct 15 '13 at 20:27
add comment

OK here's the web crawler I referred to in my original post:

Disavow Web Crawler (disavow_link_crawler.zip)

This tool will loop through your sample links file (available from Google Webmaster) and search for the strings defined in the 6 boxes visible within the crawler interface. It's pretty basic but it does give you a way to isolate spammy links, in my case an SEO company created dozens of links with varying descriptions appended so I used this tool to isolate the links and build a disavow sample file.

Link crawler interface

When you run the crawler it gives you a live output to the screen showing each link crawled. Any that yield a match will be highlighted in red and appended with M followed by the matching pattern number e.g. M1. When the crawler has finished it will give you a count for how many times each match pattern was triggered and it will build a domain-based (see below) disavow list.

Output when running

Domain based disavow list

This program will disavow at the domain level. This means that output will look something like this:

domain: 0directory.com
domain: 3dir.co.uk
domain: 4seeking.com

This may not be suitable for all cases but for me there was very little risk in disavowing the whole domain. The chances of inadvertently targeting quality links elsewhere on the domain was close to zero.

If you do want the very targeted url based disavowment then you'd need to modify the code slightly.

Retrieve and upload the Disavow file

Once the crawler completes it will write a file disavow.txt in the root of the application. You can grab this, definitely give it the once over though (I can't guarantee the program will not doing anything dumb :)) and then upload it to the Google Disavow tool.

Make sure to read the warning regarding usage of the disavowment tool, it can if used incorrectly have some pretty adverse effects on ranking.

If anybody wants to take this further and build on what I've done here, please do!

share|improve this answer
add comment

Not the answer you're looking for? Browse other questions tagged or ask your own question.