I read that the more plugins you have, the more bloat that needs to be loaded. Therefore I am prudent in adding plugins to my site.
What are the pros and cons of using google analytics with a plugin (like Monster Insights) vs without a plugin?
Hard coding edits into core theme files goes against WordPress best practices. Each time the theme receives an update, you are going to need to re-add the tracking code. Also if the theme were to be changed to a different one, the tracking would also have to be reinstalled.
Use a plugin. There are a few advantages to some of the plugins available. Some allow for easy blocking of your own and other site admin, editors etc visits so those visits don't inflate the site data. Easy config of event tracking for things like outbound links, file downloads and so on. You would need to read up on a particular plugin as to what it offers.
If you are familiar with php, js and how WP loads scripts etc, you could also create your own custom functions plugin, which wouldn't have all the easy interface/options of one of the developed plugins for GA tracking, however it would be a quick way of simply adding the tracking code to your site without making edits to your theme files. http://justintadlock.com/archives/2011/02/02/creating-a-custom-functions-plugin-for-end-users