Skip to main content
New
Stack Overflow Jobs powered by Indeed: A job site that puts thousands of tech jobs at your fingertips (U.S. only). Search jobs

Questions tagged [meta-robots]

The tag has no usage guidance.

3 questions with no upvoted or accepted answers
Filter by
Sorted by
Tagged with
1 vote
0 answers
209 views

Should Feed/RSS Be Blocked from Robots.txt?

Should Feed/RSS be blocked via robots.txt? Disallow: /*feed* Disallow: /*rss* If yes, is this the correct way to apply? If you are sending Feed/RSS somewhere or if you have a subscription system, of ...
gfo95's user avatar
  • 41
1 vote
0 answers
24 views

Can you specify the max-snippet length in a X-robots-tag header?

The specs by Google does not address this issue. So, I would like to know what is the x-robots-tag equivalent for: <meta name="googlebot" content="max-snippet:20"/> Is it ...
Jesske's user avatar
  • 11
0 votes
0 answers
25 views

can i deeplink my product pages which are not allowed to be crawled by robots

I have a set of pages which i have blocked by the robots and i don't want them to appear in search results. But if i am targetting users on this page via email/ad campaigns i want a possibility of ...
abhinsit's user avatar
  • 1,714