I don't think there's a whole lot of official information on the web about brand mentions in relation to spam. I also don't think there's a disavow tool for brand mentions, though perhaps there should be.
Because Google's language algorithm uses word association to determine what a word means, when it sees your brand on pages of a certain topic it's very likely that it will associate your brand to that topic. This is similar to how Google associates your site's niche to the links pointing to your site from other website's niches.
So yes, I personally think it's quite possible that the brand mentions of your website on those sites could negatively effect your SEO. I don't think it would have as big of an impact as spam backlinks, but the association is certainly a possibility.
There have been sites who have had competitors try to derank them by spamming them with bad backlinks. And it's worked. I haven't heard of anyone receiving spam brand mentions in a way that negatively effects their SEO until now. But I wouldn't be surprised if that's actually possibly hurting you. I wish I knew of a way that you could disavow the brand mentions from those sites, though I haven't heard of such a tool.
Either way, if those sites are low ranking, low trust then the spam on there shouldn't impact you too much if your site has high quality content or high trust. The best thing that you can do is keep pushing out top of the line content and getting good, solid backlinks from reputable websites.
Google also did also implement a fairly impactful algorithmic update in March. The update was largely about quality and improving the rankings of websites with good content. It's possible that your deranking was a result of the algorithm update. There's a lot of climbers and a lot of fallers when Google releases updates like these.