How can I stop pagination URLs from getting indexed? [duplicate]

Question

I want to prevent pagination URLs from being crawled or indexed anywhere.

When I check site:example.com, the search results looks like this.

I have not listed this type of URLs in sitemap.xml

How do I stop this from happening?

Google doesn't use sitemaps to decide what to index. Google probably won't index everything in your sitemap. If they find other pages on your site with their crawler, they will probably decide to index some of those. Sitemaps just don't work like you are expecting them to. In fact, they have very little use for SEO at all. See The Sitemap Paradox — Stephen Ostermiller, Nov 20 '19 at 15:38

Evgeniy · Accepted Answer · 2019-11-20 14:01:48Z

You should firstly decide, what to do at first: prevent indexing, or prevent crawling? Both at the same time isn't possible.

Lets assume, your pagination is already indexed. Than you should firstly de-index it, and then prevent crawling. Do it on the following way:

1.1. If you can, add to each paginated page the meta tag <meta name="robots" content="noindex".

1.2. Otherwise add to your htaccess (in case of Apache 2.2) or server configuration file (in case on Nginx) a rule like:

SetEnvIf Request_URI "/page/.*" NOINDEXFOLLOW
Header set X-Robots-Tag "noindex, follow" env=REDIRECT_NOINDEXFOLLOW

1.3. For Apache 2.4 use

<If "%{REQUEST_URI} =~ m#/page/.*#">
Header set X-Robots-Tag "noindex, follow"
</If>

1.4. Note, if use Wordpress, set this rule on the beginning of htaccess file.

1.5. Create an additional sitemap with paginated page and submit it to GSC - on this way Google understands faster, what to do.

1.6. Than monitor SERP. After you realize, paginated pages aren't no longer in SERP, than:

the 1 2 3 order is important because if you Disallow first, robots wont be able to crawl pages and find the information you want to deindex them so they won't 🤔 — gael, Nov 20 '19 at 15:27

1 Answer 1