Googlebot has been able to follow links in JavaScript since 2007/2008:-
We already do some pretty smart things like scanning JavaScript and Flash to discover links to new web pages
They have been executing JavaScript since at least 2009 (I imagine they learned a lot from building Chrome), and Matt Cutts has publicly confirmed that they can follow javascript links, execute scripts and submit forms.
"For a while, we were scanning within JavaScript, and we were looking for links. Google has gotten smarter about JavaScript and can execute some JavaScript. I wouldn't say that we execute all JavaScript, so there are some conditions in which we don't execute JavaScript. Certainly there are some common, well-known JavaScript things like Google Analytics, which you wouldn't even want to execute because you wouldn't want to try to generate phantom visits from Googlebot into your Google Analytics".
The upshot of this is that, no, it isn't hurting your SEO, provided your JavaScript is well formed and relevant. I'd recommend using fetch as Googlebot in webmaster tools and turning on server logging for the Googlebot and watching where it's going if you really want to check.
The first link is to a thread at webmaster world, it's well worth the read as is the third link to a Matt Cutts interview. I would also read this question 'Does the Google spider render JavaScript?''Does the Google spider render JavaScript?'
