I have an 11 years old lyrics website and it has the biggest archive for my region. Before September 4, 2013 it had nearly 16.000 unique visitors per day and was in the first 3 SERP results for main competitive keywords. On Sep 4, 2013 it became 12 for a day, then continued with around 7000 unique visitors/day.
I havent get any manual webspam message on Google Search Console and my website shows in results when I search it with it's name. Not banned completely.
I thought it could be because of "Hummingbird" and I was adding a hidden link (to my site) to lyrics for avoiding copying and for getting backlinks when copied. I thought that could be the reason and removed it after this huge visitor decrease.
It looks like I had an algorithmic penalty, but after 4 years, my web site still not in first 500 results for these main competitive keywords (but my website is listed as 1 on Yandex for same main keywords). My visitor count also hasn't increased. In fact newer sites with much less content/archive get better rankings.
I tried many things but couldn't solve this. The last thing I suspect is that I also have video pages integrated with YouTube API (separate page for each lyrics and artist page). My website queries their API and shows the results directly and then opens videos directly from YouTube in a popup (with embed code). Could this be the reason for the penalty?