Take the 2-minute tour ×
Webmasters Stack Exchange is a question and answer site for pro webmasters. It's 100% free, no registration required.

I had a couple private files in a directory on my school folder. You could see that the files existed by going to myschool.edu/myusername/myfolder, but trying to access the files themselves via myschool.edu/myusername/myfolder/myfile.html returns a 403 error.

And yet Google somehow managed to grab the contents of those private files and store them in its cache! How is this possible? [I've since removed those files, so I'm just curious how Google managed to do this.]

share|improve this question

migrated from stackoverflow.com Dec 4 '10 at 19:01

This question came from our site for professional and enthusiast programmers.

2  
This belongs on Webmasters –  RobertPitt Dec 4 '10 at 18:57

1 Answer 1

The most probable reason is that the pages won't return a 403 header.

You can check that using the Web Developer Toolbar in Firefox or Chrome. The tool is located under "Information" -> "View Response Headers".

Also, the way I create my error pages is:

  1. I create some dummy error page. Let's say 403.php.
  2. I create an actual error page. For example error403.php.
  3. On the dummy error page, I put the following code: <?php header("Location: /error403.php",TRUE,301); ?>
  4. In my .htaccess, I put the following:

    Options -Indexes

    ErrorDocument 403 /403.php

This adds all the redirects in a proper way and makes me sure I'm getting some juice from my error pages.

This can actually be extended in an extremely cool way if your website has a search engine which uses GET requests.

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.