Hot answers tagged

9 votes
Accepted

How does "Noindex:" in robots.txt work?

Google used to unofficially support a Noindex directive in robots.txt, however in 2019, they announced that the directive will no longer work. Here is what Google's John Mueller says about Noindex: ...
9 votes
Accepted

Do noindex,follow pages pass pagerank / link juice?

Yes. Eric Enge: Can a NoIndex page accumulate PageRank? Matt Cutts: A NoIndex page can accumulate PageRank, because the links are still followed outwards from a NoIndex page. Eric Enge:...
  • 2,476
9 votes

Do search engines still crawl a noindex page

Yes, Google still crawl webpages that have noindex tag. But if you have same content on two different webpages and one URL contain noindex tag, while second does not, then you should not worry about ...
  • 5,487
8 votes
Accepted

Does the Title and meta description for noindex pages influence SEO ranking in any way for the rest of the indexed pages?

No, it does not influence in ranking, because that pages is not indexed at all, also it does not harmful for your site in some ways, but if you are placing too many noindex tags, then those pages will ...
  • 5,487
8 votes
Accepted

Ok to use meta noindex on 404 pages?

There's no problem having a meta noindex tag on the 404 page, to prevent 200 OK responses being indexed. If this was a PHP page then you could obviously just send a 404 Not Found header as part of the ...
  • 41.6k
8 votes
Accepted

How do I "noindex" a text (.txt) file?

The solution is the same as for X-Robots noindex a folder of PDFs and Prevent XML sitemaps from showing up in Google search results. You use the X-Robots-Tag HTTP header rather than a meta tag. ...
8 votes
Accepted

Why does it cause issues to use robots.txt and a meta robots tag at the same time?

The robots.txt standard controls whether the bot can view foo.html under any circumstances. Just because the bot sees a link to it from a different site doesn't give the bot permission to sneak a peek ...
7 votes
Accepted

Do search engines still crawl a noindex page

As Goyllo has already stated, search engine bots will crawl pages that have a noindex meta tag. If you think about it, they need to crawl the page in order to see the noindex meta tag in the first ...
  • 41.6k
7 votes

URLs with 'NoIndex` in robots.txt are being indexed by Google

"noindex" directives should not be used in your robots.txt file, instead a noindex meta tag should be added to any pages that you don't want indexed in Google. A NOINDEX tag looks like the below and ...
6 votes

URLs with 'NoIndex` in robots.txt are being indexed by Google

Note that Noindex is not part of the original robots.txt specification. Google supported it as experimental feature (see: How does “Noindex:” in robots.txt work?), but it’s not clear if that is still ...
  • 21.5k
5 votes

No index for an entire site

Nick's answer to add robots.txt might not stop Google from index, it just says stop crawling but Google might index - official link here. Adding meta noindex is too risky if you are working with a ...
  • 526
4 votes
Accepted

Robots.txt vs. Noindex Tags

Good catch! If you're blocking a page with robots.txt then crawlers will not able able to read the noindex meta tag. In these cases you should send the x-robots-tag HTTP header either via server-side ...
  • 85.8k
4 votes
Accepted

if an HTTP page can be cached by proxy should I put "noindex" in the HTTPS version or not?

A canonical link element is an HTML element that helps webmasters prevent duplicate content issues by specifying the "canonical" or "preferred" version of a web page. By default there can be numbers ...
4 votes

Will meta setting Meta="NOINDEX" eventually remove page from Google index?

I am not sure if your theory is good because it is based on a very big assumption that your new website will take the credit for your old site after deindexing. Except rel="canonical", you don't have ...
  • 1,316
4 votes
Accepted

How to correctly remove MULTIPLE parameters from the Google index using htaccess and canonical / noindex?

You only really need to set the rel="canonical" header. This should be sufficient in ensuring only the canonical URL (ie. the one with no URL params) appears in the SERPs. Setting a noindex robots ...
  • 41.6k
4 votes
Accepted

Is there differences between deleting low quality articles vs no-indexing low quality articles SEO wise?

You can take either path here. What matters is how you go about it. Let's say you decide to delete those articles. That should be fine. Just make sure you 301 redirect those URL's to something better ...
4 votes

How to prevent google index for a specific subpath?

You are sending Google two conflicting commands: you are telling them to NOINDEX some pages but then you are also preventing them from reading the NOINDEX header by blocking them from crawling these ...
4 votes
Accepted

Deindex a site through Google Search Console without access to the site's server to make changes?

You Can Submit a Removal Request Submitting will remove URLs for 6 months, this will buy you some time until you're able to put a permanent solution in place. In your case, you might find the prefix ...
  • 5,467
3 votes

How do search engines behave when encountering noindex?

What I don't know is whether it also blocks the URL from showing up in search results, or just that particular content with that URL. If you have a noindex robots tag then the URL should not appear ...
  • 41.6k
3 votes

What are the reasons for using the meta tag "noindex, follow"?

Faceted navigation / search filtering allows visitors to narrow their search for specific products. Such navigation features generate crawl paths that are of no real value to a search engine but are ...
3 votes
Accepted

How to do SEO for a business listing website that contains temporarily available content

It is very hard to get local directory sites ranked in search engines these days regardless of whether or not the content is temporary. See this parody letter purportedly from Google, but really ...
3 votes

X-Robots-Tag: noindex on an HTTP 302 response to remove subdomain

Google considers the status code before looking at everything else, so the X-Robots-Tag will be ignored. We did some test a year ago with a client and this was not working. The strategy that we used ...
  • 181
3 votes
Accepted

Is it possible to regain a site's traffic after losing some to a transfer / merge?

If you are redirecting old pages to the new pages that have the same content as the redirected pages (and ideally the same URL structure), then patience is needed. I suggest that you look at the ...
  • 733
3 votes

Do noindex,follow pages pass pagerank / link juice?

Google recently (December 2017) said that noindex,follow pages end up not not passing link juice. Initially link juice will get passed, but eventually the links on the noindex page will be ignored....
3 votes
Accepted

should you let search result page to be indexed

According to Google's Webmaster Guidelines: Use robots.txt to prevent crawling of search results pages or other auto-generated pages that don't add much value for users coming from search ...
  • 2,299
3 votes

Should I set NOINDEX header for my JS, CSS and image files?

Nowadays, don't worry about it. Although we don't set the x-robot http header noindex for static file css, js, Googlebot know what file should be indexed or not. Except image, because the images can ...
3 votes

What is the proper SEO handling of pages appearing in popups using IFRAMEs?

The noindex should be enough to stop it getting indexed. With regards to iframes and linking, I can't actually find any guidelines from Google. In fact, all they seem to officially say on the entire ...
  • 1,925
3 votes

Is it 100% safe to add NOINDEX tag to any duplicate/copy content issues?

Both rel=canonical and noindex has their own purposes. rel=canonical should be used when you want to give weightage of the page to a page from where the content originated. The seo juice in this case ...
  • 1,694
3 votes
Accepted

Is Google starting to ignore link directories such as brownbook

Generally speaking, Google does not like directory sites too much. Some perhaps. However, most are generally either low quality or spammy or even replicated information that can be found everywhere. ...
  • 32.6k
3 votes
Accepted

SEO put no-indexed pages in sitemap or not?

I'm kind of wondering if I've missunderstood your question, but... there is no point putting "noindex" pages in your XML sitemap. An XML sitemap advertises the pages you want indexed. So, including a ...
  • 41.6k

Only top scored, non community-wiki answers of a minimum length are eligible