I am currently linking related and text (caption) in the following way:
<a href="/link"><img src="image.jpg" ... />This is caption</a>
Meaning I combine both and text inside the same tag. Is that the right way to do it, syntax and SEO wise?
That's perfectly fine. It's usable, accessible, and (not coincidentally) search engine friendly.
(You might consider using the alt attribute of the image tag to describe the image for users with screen readers or for when the image fails to load. But that doesn't directly affect your question as the caption essentially serves as anchor text for that link).
as I know, google loves text links better then image links. But I don't know, what is the case with mixed links like yours, but I would use the background-image CSS property of the A tag instead of using img tag in the link.
This can cause a problem, if you print that page, because background images won't be printed as default.