Questions tagged [meta-robots]

The tag has no usage guidance.

Filter by
Sorted by
Tagged with
0
votes
0answers
15 views

Which is the best meta robots tags? [on hold]

Recently I found 2 more meta robot HTML tags, but I am not sure about which is the best one: <meta name="robots" content="follow,index" /> <meta name="robots" content="max-snippet:-1, max-...
2
votes
1answer
59 views

What are the potential side effects of the “nosnippet” robots meta tag?

Just recently I noticed our Search Console CTR drop from 4% to 2%. I noticed that we had an ever increasing number of impressions, but despite the increase in impressions, our clicks had decreased. ...
0
votes
2answers
117 views

Disallowing URL parameter in robots.txt

I have important URLs on my site to which I’d like to append temporary parameters. For example: example.com/tool/input will have the parameter example.com/tool/input?foo input in the URL is ...
2
votes
1answer
93 views

Are there only 12 directives for <meta name=“robots” />?

As far as I can tell, in 2017, the <meta name="robots" content="index, follow"> accepts the following directives in the content attribute: index follow all (equivalent to content="index, follow"...
3
votes
1answer
272 views

Can the content attribute of <meta name=“robots” /> be left empty?

Is <meta name="robots" content="" /> a legitimate meta, exactly equivalent to: <meta name="robots" content="index, follow" /> <meta name="robots" content="all" /> [nothing] Or ...
1
vote
3answers
541 views

does google crawl/follow link on a page with meta robots noindex

We have added following meta tag on some of our pages: <meta content="noindex" name="robots"/> Does google crawl the content and links on this page and just don't show it on the google search ...
1
vote
1answer
1k views

Use robots.txt to prevent privacy policy, terms and conditions, and guarantees from being crawled and indexed by Google

I need to block pages such as privacy policy, terms and conditions, and guarantees from being crawled and indexed by Google. They might be unoriginal. I use them for my multiple websites and they ...
2
votes
2answers
34 views

Method for removal of high number of spammy links

I haven't been able to figure out the right combination for deindexing spammy links from search results. On one hand, we have the Google URL remover which removes links temporarily (they may show up ...
2
votes
1answer
351 views

Help with robots meta tag and X-Robots-Tag

I have two questions about robots meta tag and X-Robots-Tag: If a page has both X-Robots-Tag HTTP header and robots meta tag, can this cause problems? For example: On HTTP response headers: X-...
0
votes
0answers
22 views

can i deeplink my product pages which are not allowed to be crawled by robots

I have a set of pages which i have blocked by the robots and i don't want them to appear in search results. But if i am targetting users on this page via email/ad campaigns i want a possibility of ...
2
votes
1answer
147 views

Meta tag robots noodp

Does Google still crawl a page when this is used? <meta name="robots" content="noodp"/> And when should this be used? Our robots.txt is set to index all.
1
vote
2answers
817 views

difference between `<meta name=“robots” content=“nofollow” />` and `rel=“nofollow”`

Is the meta tag <meta name="robots" content="nofollow" /> The same as rel="nofollow" in a link (apart from the fact the the meta tag would be page wide rather than on a per link basis) Or ...
4
votes
1answer
138 views

URL in sitemap with temporary/dynamic noindex

Our scenario is we've got multiple pages set up for category-based browsing, such as example.com/products/boardgames A list of the canonical URLs for the categories is in the sitemap, with a frequency ...
1
vote
2answers
202 views

Robots Meta Tag and query strings

I am trying to find out something about this tag: <META NAME="ROBOTS" CONTENT="NOINDEX, NOFOLLOW"> If I have a dynamic page like these example.com/product.aspx?ProductID=123 example.com/...
0
votes
1answer
282 views

What are the reasons for using the meta tag “noindex, follow”?

Besides using the meta tag for hiding, say, the sitemap to prevent your site appearing spammy, are there any other situations for using noindex, follow?
4
votes
2answers
1k views

If I dynamically add a robots meta tag (via JS) will Google and other search engines recognize it?

I want to have a chunk of JS in the middle of my page insert a meta-robots element into the <head> based on some arbitrary condition. Will Google and other search engines recognize this tag the ...
5
votes
1answer
544 views

What is the default behavior of robots if there is no robots meta tag?

Assuming I have a page without a robots.txt and without the robots meta tag. What is the default behavior of robots? Would they index the page and follow the links as in: <meta name="robots" ...
1
vote
1answer
400 views

AdSense is adding a meta robots tag with noindex,nofollow - would this negatively affect my SEO?

When I add AdSense code to my webpage, I noticed that AdSense has a meta robots tag like below: <meta content="noindex,noarchive,nofollow" name="robots"> My page has no other meta robots tags. ...
3
votes
2answers
4k views

Robots meta tag with “noimageindex”

I have some doubt regarding noimageindex value in meta robots tag. If I add this tag on my page say http://www.example.com/somepage/someaction.php and on that page the images are served from another ...
1
vote
1answer
49 views

meta robots for logged in vs. not logged in users

On a site I'm working on, The homepage has generic content for visitors who aren't logged in. When a user is logged in, the content shown is personalized to them and could even be considered private. ...
4
votes
1answer
9k views

What does “all” mean in meta robots tag?

Several websites have the following meta tag aimed at search engines like Google, Bing, etc.: <meta name="robots" content="all" /> What does it do?
2
votes
3answers
422 views

Should I use both Robots.txt and META NAME=“ROBOTS”?

I am currently using the Robots.txt file (the standard one of Joomla, to say). Now I have found that there are also specific HTML meta tags that specify desired robot behavior. Is there a reason to ...
8
votes
2answers
1k views

Which meta “robots” tag gets preference?

My wife works at a high school in Germany. I recently noticed that it's extremely hard to find that school's homepage using Google. I looked at the source code of the page and I believe I've found the ...