From what I understand, in NTFS the free space preference algorithm is
such that if it can, windows will fill up the pieces of free space in
order on the disk from the beginning of the partition itself rather than
go over to the great swathes of free space further down and plop the
file there. Quite stupid, but that's what we have and what we need to
live with.
In it's defense, windows has no idea how much each file will grow (or
shrink) in the future so it cannot preallocate space to it under normal
circumstances and steal that space from other files which may need it in
the future. This preallocation may not be an issue when the disk has a
lot of free space left, but what when it begins to get filled up? Even
the MFT fragments once it's outgrows it's preallocated zone, or disk
space begins to get full and other files encroach into it's zone.
Fragmentation is therefore a dynamic problem and needs a dynamic
solution w.r.t disk volumes that see heavy I/O activity. That's why
automatic defragmenters are becoming the preferred solution on the
server (and even workstation) side of things in the enterprise segment;
because unlike the older occasional scheduled/manual defrag processes,
they are a continuous approach to tackling fragmentation. The icing on
the cake is that auto defragmentation is also largely autonomous and
the admin's workload is greatly reduced.
As long as there is windows in the current form, we are stuck with
fragmentation. Maybe we ought to ask MS to create a new non-fragmenting
filesystem for Vienna (Yes please, and large fries with that order)
