Dead URLs from the previous CMS on this domain is displayed in SERP, most likely due to people still trying to access them through bookmarks or browser history. This traffic is picked up by Google Analytics, and probably causing these dead URLs to be prioritized because they are popular/often accessed.
This is all perpetuated because Wordpress is not responding with 404/301 when these ghost URLs are accessed.
Is there a way to force Wordpress to respond with 404/301 when visitors are accessing non-existing pages?
Right now visitors get served an 'empty' page, displaying only header and footer – no real content.
EDIT: Reworked the text in light of new information to clarify the problem.