I have directory that contains 400,000+ files. Potentially the directory could easily contain ~30,000,000 files.

  1. Is that a good idea or should I rather chop it up in smaller directories like this:

  2. What size should I make the smaller directories? Would 100,000 files in each directory be a good idea?

  • 1
    What is your main concern..load times, directory limits, SEO, directory structure? Commented Sep 23, 2016 at 22:24

2 Answers 2


I am going to give this a shot.

From a filesystem point of view:


  • Maximum number of files: 268,173,300
  • Maximum number of files per directory: 216 - 1 (65,535)
  • Maximum file size: 2 GiB - 1 without LFS, 4 GiB - 1 with


  • Maximum number of files: 232 - 1 (4,294,967,295)
  • Maximum file size
    • Implementation: 244 - 26 bytes (16 TiB - 64 KiB)
    • Theoretical: 264 - 26 bytes (16 EiB - 64 KiB)
  • Maximum volume size
    • Implementation: 232 - 1 clusters (256 TiB - 64 KiB)
    • Theoretical: 264 - 1 clusters


  • Maximum number of files: 1018
  • Maximum number of files per directory: ~1.3 × 1020 (performance issues past 10,000)
  • Maximum file size
    • 16 GiB (block size of 1 KiB)
    • 256 GiB (block size of 2 KiB)
    • 2 TiB (block size of 4 KiB)
    • 2 TiB (block size of 8 KiB)
  • Maximum volume size
    • 4 TiB (block size of 1 KiB)
    • 8 TiB (block size of 2 KiB)
    • 16 TiB (block size of 4 KiB)
    • 32 TiB (block size of 8 KiB)


  • Maximum number of files: min(volumeSize / 213, numberOfBlocks)
  • Maximum file size: same as ext2
  • Maximum volume size: same as ext2


  • Maximum number of files: 232 - 1 (4,294,967,295)
  • Maximum number of files per directory: unlimited
  • Maximum file size: 244 - 1 bytes (16 TiB - 1)
  • Maximum volume size: 248 - 1 bytes (256 TiB - 1)


From a functionality point of view:

Keep in mind that on Linux if you have a directory with too many files, the shell may not be able to expand wildcards. I have this issue with a photo album hosted on Linux. It stores all the resized images in a single directory. While the file system can handle many files, the shell can't.

From a server speed point of view:

Too many files in one directory can cause load times to increase by seconds. Having too many directories can also increase load times. (Your server specs play a role in this)

From an SEO point of view.

I can understand not having proper image names for security reasons (assuming a user uploads photos and you have a rewrite in place) But you really should give a content relevant (sub)directory names for better search ranking and the ability to partially remove URL's to get to another spot on your server. (ie photos/outdoor/landscape/mountains)

In the end there is no one fits all but you can make a better informed decision based on the aforementioned.

  • 1
    It has been a while, and for the record, I do not know where the limit is, however, too many files in a directory will make it nearly impossibly slow for the file system to address any file in the file index. I experienced this years ago. Even if you do not hit that limit, other tools can consistently fail such as FTP. I have one directory that is huge but not that huge and FTP of any flavor or stripe will have a conniption and will fail every time. It is better not to push the limits. Cheers!!
    – closetnoc
    Commented Sep 23, 2016 at 23:31
  • Agreed, not pushing the limits. I personally think a couple of thousand is too many. Even on my mac if I open a folder with 2,000 photos, it starts to struggle. Commented Sep 23, 2016 at 23:54
  • 1
    For the record- I like your answers! Cheers!!
    – closetnoc
    Commented Sep 24, 2016 at 0:07
  • I get worried about commenting off-topic but I do want to say I greatly appreciate your kind words! Cheers!! :) Commented Sep 24, 2016 at 0:35
  • The max number of files in a directory on FAT32 ("65,535") is assuming you are using short 8.3 filenames. If you are using long filenames (LFNs) then this number can be greatly reduced, since each 13 byte filename chunk is stored in a separate directory entry
    – DocRoot
    Commented Sep 27, 2016 at 14:35

I've tried to have directories with tens of thousands of files on ext3 and ext4 Linux. It eventually gets to the point where listing those files is prohibitively expensive. It can take a few seconds for ls to complete.

I would try do design for a max of 10,000 files per directory.

One way of doing so is to use the first couple characters of the file name as a directory. The git source control system takes this approach. See Why does git store objects in directories with the first two characters of the hash?. Git limits the number of hash revisions per directory to 6700.

  • I did a test with 10.000 and 1.000.000 files: 'ls' work time in linear dependence. Access time to 1 file is the same. So there is no magic number like 10.000 - just a desired time for directory list operation. So you can split 1m to 10k, but if at the end you need to get a whole list of files on a whole disk it doesn't matter.
    – Yura
    Commented May 11 at 13:31

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.