You probably where having issues with ghost referrals or with third party websites crawlers also known as bots. It is claimed that ghost referrals are a serious problem and will mess up with your data. On the contrary, some legitimately bots will periodically visit your website to extract data from you to feed or update their records about you.
You should identify if you are having issues with referrals or if there is a recurring visit to your site from any particular bot within the affected time frame.
Searching in google for ghost referrals will yield many possible solutions or how to fix it. For those bots that you can identify you should decide either to block them via robots.txt or create filtered view in GA. However, be careful not to block "good" bots from services that you are running and that might require to visit your website now and then.