I'm thinking of creating a site where users are allowed to upload content that can then be purchased by other users. My question is how do I protect against someone uploading something that isn't theirs? Do I have to do that with a team of people or are there methods that can be implemented to protect against this. How does Youtube know that someone has uploaded something that violates a copyright?
This depends a lot on your local copyright (and other) laws, but generally it should be sufficient to ensure that you:
If you and your site are located in the U.S., there's a particular set of requirements laid out in the Digital Millennium Copyright Act that, if followed, qualify you for DMCA Safe Harbor protection. These requirements are broadly similar to the general rules above, but do include a number of specific details that you need to be aware of and comply with. Other jurisdictions may have similar laws, or may simply rely on general legal principles such as intentionality and due care to determine whether or not a service provider might be considered liable for infringement initiated by users.
All that said, the first thing you should do, before embarking on a project where you're concerned about the risk of legal liability, is consult a lawyer. I'm not one, and this post is not legal advice.