Suggestion: Use noindex tag if your tags and categories are not related.
I will recommended to use noindex meta tags on your tags webpages, the benefits is google will still crawl that webpages and pass the juicyrank easily.
And remove the tags links from your blog post, so your juicyrank will pass to only your important webpages.
Suggestion: Use 301 redirection if your tags and categories are similar.
301 and Canonical both will do something, but as mike suggest Google and major search engine follow 301 redirection very well. And I recommended 301 redirection only if tags and your categories is somehow related.
Suggestion: Use 410 error page, if you want your website clean (Not mean by speed)
When you remove such a tags from your website, then all of the tags links will automatically deleted from your blog, so your juicyrank will pass to other webpages, so 404 or 410 is not bad at all. By removing such webpages will make your website clean, so you don't need to focus on tags further more.