I think you get the backlink to your site, you may get some rep if you're guest blogging on a highly trafficked or influential site, but that may be all you end up getting out of it.
It may depend a lot on the rules that you have to follow on the host site. Maybe they'll let you backlink to your site and provide a follow link to your twitter account? Maybe they'll let you share in some of the proceeds from any ad activity on that page?
It would probably help to start out by determining what you want to get out of the process. If you want to simply amass a lot of backlinks to your site, commenting on a lot of similar blogs to your own may be just as effective as guest blogging. If your blog does not get a lot of user feedback on posts you publish and you are keen on finding out how effective your writing style is, guest blogging on a more highly trafficked site could be good for you.