0

We want to block the pagination URLs from Google & other search engines, is this possible without blocking in robots.txt?

  • Do you want to block crawling or indexing? – unor Mar 13 '18 at 13:09
  • 1
    You say in a comment "we are not using any parameters (&, ?) for pagination". Please edit your question and give us more information then. What are examples of your pagination URLs? – Stephen Ostermiller Mar 13 '18 at 13:44
1

You can tell Google which pages are pagination here:

https://www.google.com/webmasters/tools/crawl-url-parameters?hl=en&siteUrl=

Enter in the parameter of your pagination pages and you can select the dropdown that says that these pages paginate.

If you don't currently have a parameter that you can use for your paginated pages you can add them by 301 redirecting your current pagination to the parameter URL.

You can also add rel=canonical to your pagination pages though it's not as good of a solution as the crawl url parameters feature in search console.

  • 1
    "If you don't currently have a parameter that you can use for your paginated pages you can add them by 301 redirecting your current pagination to the parameter URL." - by "301 redirecting"?! That sounds like a very bad idea! – DocRoot Mar 14 '18 at 17:03
  • @docroot so true, keep it to 302 unless you're sure of what you're doing. – Michael d Mar 14 '18 at 20:09
  • It was the external "redirect" that I was concerned about, not the status code. – DocRoot Mar 15 '18 at 15:08
1

If you're using an SEO plugin (for WordPress), you can tick the checkbox to block Google from indexing pagination. You can usually see this in the General settings (All in one SEO) and in the advanced feature tab if it's Yoast.

Or perhaps, you can block Google bots from accessing these pages at your robots.txt file:

Disallow: /*?page=
Disallow: /*&page=
  • we are not using any parameters (&, ?) for pagination – Sandhiya Rao Mar 13 '18 at 3:00

Not the answer you're looking for? Browse other questions tagged or ask your own question.