4

I am trying to include the following in my response header to all resources in multiple directories (dirone/dirtwo/dirthree/direfour) from an Apache server.

X-Robots-Tag: noindex, nofollow

In my httpd.conf file I have added the setting to the dirone underneath my main directory declaration.

It is not producing the additional item in the response header. Do you see what I am doing wrong?

<Directory />
    Options FollowSymLinks Includes
    AllowOverride None
</Directory>

<Directory "/web/sitename/docs/dirone">
    Header set X-Robots-Tag "noindex, nofollow"
</Directory>

Additional information. I am moving away from the robots.txt approach, we do not use .htaccess files and I want the header in all resources in those directories and their sub-directories to have X-Robots-Tag: noindex, nofollow in them as well.

Thanks for your help.

Improve this question
3
  • Just to confirm, is /web/sitename/docs/dirone a physical directory (or symlink) on the filesystem?
    – MrWhite
    Dec 14, 2015 at 1:17
  • It is the physical directory
    – wall-nut
    Dec 14, 2015 at 4:45
  • 1
    The syntax above did work once the /dev/ was added to the path. Thanks all.
    – wall-nut
    Dec 14, 2015 at 15:33

2 Answers 2

Reset to default
1

I think the problem is that the Header directive shouldn't go in a Directory section. Instead it should go with the other configuration for sitename and use a Files directive.

So in your sitename configuration (even in your .htaccess) you should use:

<Files "dirone/*">
    Header set X-Robots-Tag "noindex, nofollow"
</Files>
Improve this answer
4
  • The Header directive is permitted in <Directory> sections and this can certainly work. So it would seem there is something else going on here?
    – MrWhite
    Dec 14, 2015 at 1:17
  • @w3d Right, something else must be going on.
    – Stephen Ostermiller ♦
    Dec 14, 2015 at 2:18
  • 1
    Thanks. it was something else. The path I used was incorrect.
    – wall-nut
    Dec 14, 2015 at 15:35
  • @wall-nut I know it's an old question, but look at my answer and see if it's ok for you.
    – jrosell
    Sep 2, 2019 at 15:12
1

Try with the following:

SetEnvIf Request_URI ^/dirone.* NOINDEXNOFOLLOW
SetEnvIf Request_URI ^/dirtwo.* NOINDEXNOFOLLOW
SetEnvIf Request_URI ^/dirthree.* NOINDEXNOFOLLOW
SetEnvIf Request_URI ^/direfour.* NOINDEXNOFOLLOW
Header set X-Robots-Tag "noindex, nofollow" env=NOINDEXNOFOLLOW
Improve this answer

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.