From experience I can tell you it will skew your clientele toward more adult sites and away from non-adult sites. This is a natural result. Adult site owners are looking for friendly site developers while companies will shy away from a developer that does adult sites. That is a given.
As for a penalty, that depends upon the site, what happens to the site in the future, and how many adult sites that link to you that go bad. Beyond that, Google tolerates links within headers, sidebars, and footers to your own sites, however, considers site wide links such as "developed by" in footers links as link spam.
Any links intended to manipulate PageRank or a site's ranking in
Google search results may be considered part of a link scheme and a
violation of Google’s Webmaster Guidelines.
The following are examples of link schemes which can negatively impact
a site's ranking in search results:
Widely distributed links in the footers or templates of various sites
You also run the risk of any site going bad. One famous example is whitehouse.com which ultimately changed from one of the most successful adult sites to one of the worst spam sites causing all kinds of problems. It was a specific target of Google's anti-spam team and will remain as a prime example of a spam site throughout history.
This is not something I would do. I also do not recommend it.