We offer a service that requires URLs from our domain to be linked to external URLs.
About ten years ago, we discovered that this functionality can be abused by miscreants -- to link to malicious URLs. So we have multiple protection mechanisms, using Google Malware database (hourly updated) to check the domain against, and similarly checking the domain and IP on spam Relay Black Lists such as SpamHaus and SpamCop.
Yet, there are always new and upcoming domains to which people can link from our site, domains that are NOT YET in any RBLs.
Some such domains have been (ab)used by people recently, and as a result, Google has included our domain on their misguided malware list too.
Apart from the automation of RBL etc, what am I missing, what else can we do to work against spammers of this kind? Our RBL check is basically "live", but how can we capture domains that are malicious before they're in RBLs?
Any good advice on getting Google to un-list our website?
(We actually blocked Google's News website on our site too, because
news.google.com has a vulnerability where there "goto" URL can take any site behind the URL and forward the user to it...so we spotted many spammers using this functionality. Google has not fixed this weakness on their site, so we have blocked
news.google.com from being used on our site. Could Google have added us to the malware DB as a result of this, in a vindictive move?)
Many thanks for any pointers or advice regarding best practices for this kind of server protection.