You can think of the major difference this way: With both, you are paying for root access, but you'll only get complete use of the hardware with a dedicated machine.
VPSes can frequently fall victim to one of the huge problems in the hosting business: overselling. At any given moment during the day, the average customer isn't going to be using all of the available allocated CPU time, memory and I/O, so it's "safe" to pack a server as full as it can possibly be. Let's use this 24-core server as an example1. 24 cores means 24 virtual machines, right? In reality, many VPS providers would pack more than twice that number on there and not even blink.
Quality providers limit overselling, but are more expensive as a result.
Commercial VPSes are appropriate for many workloads. Besides not always having a full core available to you (unless the host guarantees one), one of the common complaints about VPSes has to do with I/O speed. As this benchmark comparison from 2009 demonstrates, available CPU use is generally spread evenly over time, while disk access speeds are all over the charts depending on the provider.
If your workload isn't bound to disk I/O, then a VPS may be a fine solution for you.
1: I am biased in favor of the hardware provider I linked over all others in such a way that a disclaimer is needed.