I would like to hide all pages of our website from being read or indexed by Googlebot, except several thousand of pages which were selected for indexing.
The site was launched in April 2015. The total number of pages on the site tends to infinity because the content is generated dynamically and based on parameters selected by the end user.
I've selected 128,000 pages for indexing; these pages are listed in sitemaps.
All other pages contain the meta tag
<meta name="robots" content="noindex"/>, and all links to these pages are tagged with
But during the first two months, these tags (noindex, rel="nofollow") had not been included in the pages. Today, the total number of indexed pages exceeds 4 million.
Googlebot continues to read unwanted pages, and according to server stats it seems that Googlebot ignores "nofollow" attributes. Why does it happen? What should be done to stop it reading and indexing these unwanted pages?
The total number of indexed pages is still increasing.
How can I request a re-scan of the previously indexed pages (so that Googlebot can re-read
<meta name="robots" content="noindex" />)?
Example of pages that should be indexed:
Pages, that should not be indexed: