bybe, that's simply not true at all.
Affiliate DO links blackball your site. I could give a million example links pointing to case studies from ePN's forum, Amazon affiliates, PHPbay, Warrior Forum and more - it's just something that can't be ignored.
You will NOT get a penalty for cloaking links (cloaking = renaming the links to adapt to your domain name). Cloaking is completely legit. Read any TOS. Look at any affiliate site out there, big or small, there's hardly a single one not cloaking links. Using methods to force people into launching an affiliate link is a different story.
Google can punish whomever they want. We saw this with Panda & Penguin -- the number of innocent sites, especially those of affiliate marketers, was staggering. Nobody is going to sue one of the biggest corporations in the world for something they're doing on their own search engine, and win. A LOT of businesses went under after Panda. Many of them did nothing wrong besides being an affiliate marketer, which Google has a vendetta against.
A completely legal way to remove your affiliate links from the equation is to separate them and put them on a NOINDEXed page. Unfortunately this going to increase the number of clicks that a customer goes through to get to your affiliate links. So, you'd have a site with nothing indexable by search engines but pure content, with the affiliate stuff on NOINDEX pages which are not being mixed into part of your site's ranking. Unfortunately, this is the sort of thing you have to do if you want to continue being an affiliate marketer on a search engine that is doing everything possible to make sure you don't succeed.