I think you answered your own question in the first statement.
Using the same anchor text for a lot of your links is not a good idea. Instead, try using something more descriptive. However, this is very common so I would not get fixated on this unless your entire page says nothing but "read more". Search engines are smart enough to recognize that "Read More" is not your primary keyword.
Based on your description this is blog type website with article exerts. Make sure that instead of using "read more" to link to your articles you hyperlink the titles AND make sure that your titles are properly optimized. Having articles that are closely related to a particular subject is also key for relevancy so "bundled" your articles by subject.
Also, I would concentrate on optimizing the following:
1) Page Title + Meta Title
2) Page Header (h1,h2)
3) Page Intro paragraph + Meta Description
Lastly, sounds to me like your SEO audit is being done using one of those free tools. They can often be misleading.
Have you checked your Google Webmaster Tools account to see what queries are coming up? If you linked your GTW account with your Google Analytics account you can look under Acquisitions --> Search Engine Optimization --> Queries. This will give you a better idea of how people are finding you in search.