Questions tagged [meta-robots]
The meta-robots tag has no usage guidance.
3
questions with no upvoted or accepted answers
1
vote
0
answers
424
views
Should Feed/RSS Be Blocked from Robots.txt?
Should Feed/RSS be blocked via robots.txt?
Disallow: /*feed*
Disallow: /*rss*
If yes, is this the correct way to apply?
If you are sending Feed/RSS somewhere or if you have a subscription system, of ...
- 41
1
vote
0
answers
24
views
Can you specify the max-snippet length in a X-robots-tag header?
The specs by Google does not address this issue. So, I would like to know what is the x-robots-tag equivalent for:
<meta name="googlebot" content="max-snippet:20"/>
Is it ...
- 11
0
votes
0
answers
25
views
can i deeplink my product pages which are not allowed to be crawled by robots
I have a set of pages which i have blocked by the robots and i don't want them to appear in search results.
But if i am targetting users on this page via email/ad campaigns i want a possibility of ...
- 1,714