Take the 2-minute tour ×
Webmasters Stack Exchange is a question and answer site for pro webmasters. It's 100% free, no registration required.

People are able to view directories / folders on my site right now. So, they could go to mysite.com/images and see the full index. To prevent this, C-Panel offers an option to set a directory / folder to "No Indexing" under the "Index Manager." Will this option allow webcrawlers to crawl / index the images?

Or, is there a simpler alternative to block access to all folders directly while still having it SEO friendly? My old server restricted direct access to folders by default. But, the new one does not. Any ideas on this? Thanks!

share|improve this question

2 Answers 2

It will only prevent crawlers who respect the no index or robots.txt. All of the major search engines like Google, Bing do respect this, but spambots for instance likely don't.

share|improve this answer

No, the previous answer is incorrect.

The "No Indexing" that you see in cPanel controls whether or not the web server shows a directory of all your files when there is no page. It is completely separate from the "no-index" used in robots.txt which stop search engines from indexing page.

e.g. If all your images are in /images/ folder, then allowing indexing means that people, and search engines, can view a page at /images/ that lists all of your images - whether they're used on your site or not. It also means they can see every file in every other folder on your server too - including .php files, etc.

Selecting "no indexing" will prevent that directory list from being shown to users, but will not effect whether the contents of the directory can be crawled by search engines. Search engines will still be able to index any pages and images you have on the site ... but they won't know about pages that are hidden or images that are on the server but aren't used on any pages.

share|improve this answer
    
Thanks for the response! What source are you basing this off of? –  Graham Mar 8 at 9:10

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.