Corruption will not occur. We tested into the millions.
The issue here will be performance (backups, random file opens, Explorer).
Many backup apps create a single stream per volume. So, if you have 1
volume of 2 million files, they will be read one at a time. 2 volumes of
1million each will be read 2 at a time, etc. If these are short-lived
(non-backed up) then this may not be a concern. You should check w/your
backup vendor to see how they do in the described scenario.
Depending on the organization (i.e. directory depth) Explorer may have a
problem. Explorer enumerates every file in the viewed directory. So, one
directory of 2 million files will take a while to open vs 2k folders of 1k
files each will open much, much faster. If you use programmatic access
(i.e. you have an app that creates and manages the files directly) then this
is of little concern. The cmd prompt can also be used (though obviously a
'dir' will take a while).
Finally, if the files are all on one volume, then there will only be 1 MFT
(mirrored, but only 1 is used at a time). The files are stored
alphabetically and are searched via a b-tree. So, for 1 million files the
longest it will take is 20 (2^20 = ~1million) lookups to find the file
(assuming it is non-fragmented or is one cluster in size). 2 million files
takes 21 lookups. Having multiple volumes would allow multiple MFTs which
would speed up the file lookup.
If by ~4k you mean that they are all (or almost all) <4k in size, then go
with the 4k cluster size (minimize wasted space w/out losing a lot of perf).
If by ~4k you mean they are mostly >4k but less than 8k, then you may want
to consider going with a larger cluster size of 8k. This will minimize the
fragmentation and HDD seeks needed to open/read the file and generally be
better perf (though you will have more wasted space). Larger cluster sizes
would likely waste too much space which, given the file count, could be
considerable.
Pat
