There is the potential for site wide impact but more often than not you will find a page specific impact occurring first. With any website with user generated content you need to invest your time in moderation or establish some community driven moderation (like Stack Exchange) in order to quickly catch out content that is of low quality and could affect your ranking. As a basic rule of thumb if poor quality content is identified and removed within a few hours it is unlikely to cause a substantial hit to your site as Google does encourage moderation of user generated content.
If we use Stack Exchange as an example there are a large number of poor quality posts that come up on any given day and yet Stack Exchange sites still rank extremely highly within Google, and the reason for this is proactive moderation, in effect every member of the site is a moderator as anyone (starting from a very low rep level) can flag content for moderator attention and beyond a certain reputation users are deemed to have been part of the community long enough to know what belongs and what doesn't belong and so they are given access to a range of moderation tools.