You can't make search engines forget about old URLs on your site. Once search engines find content at a URL, they will continue to periodically crawl that URL indefinitely, even after you remove the content change the URL to show a 404 error.
Instead of returning a 404 error you could change to a more relevant status:
- 301 Permanent Redirect - For the case in which there is a new page that replaces the old page.
- 410 Gone - For the case that the removed page has no replacement.
Using those statuses can make search engine crawlers back off some, especially over time. However I have old URLs on my site that have been redirecting for 20 years, and Googlebot still comes and checks on them occasionally.
You say your server is getting "slammed" by these requests because they have to go through WordPress. You can add rules to .htaccess to short circuit these URLs and serve them much faster. You can usually serve tens or hundreds of 404 requests for the same cost as one real page this way. These rules should go at the top of your .htaccess file, before any rules for WordPress.
RewriteRule ^/?html/ - [G]
RewriteRule ^/?v2/ - [G]
RewriteRule ^/?g/ - [G]
RewriteRule \.asp$ - [G]
RewriteRule \.html$ - [G]
The first three rules would show a 410 Gone
for everything in the folders you mention. The last two rules would showe 410 Gone
by extension on the URL. You should only use them if there are no more .asp
or .html
URLs left on your site.
If you can't get the rare case that it might be harming your server, you could use block those URLs in robots.txt.
Disallow: /html
Disallow: /v2
Disallow: /g
Disallow: *.asp
Disallow: *.html
Both Googlebot and Bingbot understand and honor wildcard rules, but most bots don't. The last two rules will only help for specific bots that support advanced rules.
I would only block the URLs with robots.txt if the requests cause performance problems even after trying the rewrite rules. If you disallow the URLs, search engines may still index the URLs and send some visitors to them. See Why do Google search results include pages disallowed in robots.txt? If you allow the URLs to be crawled, the bots will continue to hit the URLs, but search engines will not include them in their search indexes.