Data gathered by Google Analytics has no direct impact on your site's performance in the organic index, so, in general, you don't have to worry about this. There is one case of GA usage however, that can make things tricky - it's called Content Experiments. Rule of thumb is to always have canonical tags properly set up.
Regarding virtual page views - I recommend switching to events instead of virtual page views. As you noted in your question, each page view always provides a timestamp from which GA can calculate bounce rate and time on site/page, but this is not always a good thing. While using events, you can specify whether you wish a particular event to be treated by GA as interaction or non-interaction event. Please see the "Non-Interaction Events" section on the linked events doc page.
To give you full picture - there is also a way to stick with virtual page views if you really want to (from whatever reason). In such case, I recommend creating another property within your GA account. Use this property for virtual page views - in this way GA will gather interaction data but will not skew metrics such as bounce rate, time on site and so forth. Just remember that you need to tweak GA tracking code in order for this to work.