You have a couple of options before you. I should start out by saying that I'm also in this situation, and I let my registrar host my DNS. I believe that's the path of least resistance.
So, let's start there. One of the nice things about allowing your registrar to host your DNS zone is that they probably have some sort of fancy dancy web interface that allows you to manage all of your records. No need to muck about in configuration files with your own server. Plus, the DNS traffic is offloaded to their systems, and not yours.
One of the benefits of having DNS hosted on your own server is that you have total control. You control the TTL on your records, the server that serves it, and usually your VPS will be a little less loaded up than a registrar's name server. However, now you have to know how to configure a DNS server - and if you're using something like BIND, it requires a LOT of knowledge. Beware, though, some registrar require two name servers: unless you have a second server hanging around, this might not be feasible. Which leads us to our third option.
You could use both. Configure your DNS zones on your server and your registrar's, and use one for backup. While this is pretty fault tolerant, it's also probably an awful lot of hassle for a little extra redundancy.
Seeing as you're new to DNS, I would highly recommend keeping it simple and sticking with your registrar's servers. You're paying them for the privilege, and it saves you a lot of grief.