I am having many "Soft 404" in my site. What is/are the reason behind this error? How can I stop having this error?
We get this question periodically.
Theoretically, it is when a page does not exist and a 404 page is returned with a 200 status code in an effort to recapture an otherwise lost user.
However, with Google, thinking that it is clever, Google looked for clues such as not found on the page and marked it as a soft 404 when it may not be. Google has historically done a fairly poor job at identifying a soft 404. To defend G a bit, a soft 404 is hard to detect based upon content, however, detecting a soft 404 should not be difficult at all if you are only slightly clever at looking for other clues.
At one point, Google marked a lot of pages solely based upon term phrases found on the content page. At this point, many perfectly legitimate pages were marked as soft 404's. With so many complaints, Google experimented with various term phrases for better detection, however, it is akin to looking at the wrong end of the horse when all you have to do is cause a known 404 to see what happens or simply compare the content of several URLs that you suspect are soft 404s and see how close they are in content patterns compared to other pages.
Personally, I am not sure why they bother.
Google marks fewer pages as soft 404 these days. Personally, I think they found an acceptable level of detection without getting into the weeds of using AI (artificial intelligence) against the HTML DOM and pattern detection. Who knows?
If your page is marked as a soft 404 and it is not, then look for content clues that maybe triggering Google. I personally had to change data driven content templates and do substitutions against external data to avoid having hundreds of thousands of pages being marked as soft 404. Yeah. It was that bad!