...is it considered to be duplicate content if the site is in another
language which happens to be an exact duplicate?
Google doesn't consider the same content translated into different languages as duplicate content since
the same content in English that's translated into French is different, unlike the same content appearing twice in English, as covered by Matt Cutts here: Does translated content cause a duplicate content issue?
Does Google translate all text into one base language and then check for duplicate content?
Although Google doesn't consider a translation to another language as duplicate content, it does matter how you translate the content:
Thus by checking for various translation clues, the Googlebot can tell if Google performed the translation, and to/from what languages. If detected, then it's just a matter of reversing the To and From language pairs to translate it back into the original language in order to make the comparison.
Google prefers human translations instead that are tailored to the locale and audience, as Matt Cutts discusses here (~0:41), and might view lots of auto-translated pages as spammy.
I've also read you are to use rel="alternate" hreflang tags. Is this
to tell Google that it is duplicate content, but in a different
Exactly. As covered by Google in regards to using hreflang:
This markup tells Google's algorithm to consider all of these pages as alternate versions of each other. So you won't need to be concerned about duplicate content issues when that's used.