Prevent Google from indexing any page on my WordPress site other than the home page

Question

For months now, I have been removing old URLs that used to exist on my website going back years, using the Google Webmaster removal tool.

When I think I've removed all of the existing URLs/slugs that appear when I type SITE:WEBSITE.COM on google.com, more appear the following week and so on and so on.

How can I permanently remove all existing and future URLs and slugs that will appear via Google indexing? This is the result of used to having a WordPress site.

It's never-ending and frustrating. All I would like to see listed on Google is my homepage URL, nothing else like:

http://www.website.com/2017/07/apple-watch-series-2/
http://www.website.com/portfolio-category/web-design/page/2/

.. etc.

Answer 1

tl;dr - Please do not do this. Hop on over to the fix.

WordPress is a fantastic Content Management System (CMS), and is generally considered great when it comes to publishing content written for higher search visibility (I don't want to use the term SEO here).

What WordPress does as part of it's inherent design is for every post you publish which has a tag, or a category, it creates an archive page.

Let me explain.

You just published -

http://www.website.com/2017/07/apple-watch-series-2/

Which has a tag - apple, so WordPress will create a tag link like so - http://www.website.com/tag/apple ... and in this new "archive" page, it will keep listing all the posts which are tagged as apple.

At a content level, the first link is content of one post, the second link on the other hand is a collection of posts.

Now WordPress also has pagination and typically tends to have 7 posts per page by default (correct me if I am wrong here), which means if you have 8 posts tagged on apple, then you will have two pages of archives -

http://www.website.com/tag/apple and 
http://www.website.com/tag/apple/page/2

The content on these pages are different. The links you shared seem to be of a post and a custom taxonomy (similar to a tag or a category).

The fix

However, if you are publishing a lot of posts which have tags rarely being re-used, then you would have pages that are being created which look similar to the post. You could do a couple of things to address the concern there -

1. Ensure that your WordPress theme has a rel=canonical tag implemented.
2. If you are using a sitemap generator plugin such as Yoast SEO, then you could even stop the tag links from being generated (and included in the sitemap)
3. Remove the custom taxonomy

Hope this helps!

Answer 2

There could be many causes, why old urls still remain indexed.

• Old urls could be still internally linked on your site,
• old urls are still internally linked by non-HTML ressources, like PDF, DOC, XLSX, image maps, sitemaps,
• old urls could be externally backlinked,
• old urls could be wrongly handled, like not correctly redirected to new urls, or not correctly 404/410-ed,
• as in your example, old pages could have different pathways - some of them were removed, some remain

Answer 3

Glad you asked the question!

You should use the meta tag to prevent Google from indexing any other page then home page.

<meta name="googlebot" content="noindex" />

This tag instructs Google (but not other search engines) not to show this page in its web search results.

https://developers.google.com/search/reference/robots_meta_tag

Hope you can get it sorted out! :)