4

I am trying to include the following in my response header to all resources in multiple directories (dirone/dirtwo/dirthree/direfour) from an Apache server.

X-Robots-Tag: noindex, nofollow

In my httpd.conf file I have added the setting to the dirone underneath my main directory declaration.

It is not producing the additional item in the response header. Do you see what I am doing wrong?

<Directory />
    Options FollowSymLinks Includes
    AllowOverride None
</Directory>

<Directory "/web/sitename/docs/dirone">
    Header set X-Robots-Tag "noindex, nofollow"
</Directory>

Additional information. I am moving away from the robots.txt approach, we do not use .htaccess files and I want the header in all resources in those directories and their sub-directories to have X-Robots-Tag: noindex, nofollow in them as well.

Thanks for your help.

3
  • Just to confirm, is /web/sitename/docs/dirone a physical directory (or symlink) on the filesystem?
    – MrWhite
    Commented Dec 14, 2015 at 1:17
  • It is the physical directory
    – wall-nut
    Commented Dec 14, 2015 at 4:45
  • 1
    The syntax above did work once the /dev/ was added to the path. Thanks all.
    – wall-nut
    Commented Dec 14, 2015 at 15:33

2 Answers 2

1

I think the problem is that the Header directive shouldn't go in a Directory section. Instead it should go with the other configuration for sitename and use a Files directive.

So in your sitename configuration (even in your .htaccess) you should use:

<Files "dirone/*">
    Header set X-Robots-Tag "noindex, nofollow"
</Files>
4
  • The Header directive is permitted in <Directory> sections and this can certainly work. So it would seem there is something else going on here?
    – MrWhite
    Commented Dec 14, 2015 at 1:17
  • @w3d Right, something else must be going on. Commented Dec 14, 2015 at 2:18
  • 1
    Thanks. it was something else. The path I used was incorrect.
    – wall-nut
    Commented Dec 14, 2015 at 15:35
  • @wall-nut I know it's an old question, but look at my answer and see if it's ok for you.
    – jrosell
    Commented Sep 2, 2019 at 15:12
1

Try with the following:

SetEnvIf Request_URI ^/dirone.* NOINDEXNOFOLLOW
SetEnvIf Request_URI ^/dirtwo.* NOINDEXNOFOLLOW
SetEnvIf Request_URI ^/dirthree.* NOINDEXNOFOLLOW
SetEnvIf Request_URI ^/direfour.* NOINDEXNOFOLLOW
Header set X-Robots-Tag "noindex, nofollow" env=NOINDEXNOFOLLOW

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.