Defragmenting limitations

[tsreyb]

I have a 2nd disk (size 320GB) on WinXP SP2. It was about 80% in use
and 66% fragmented. I ran the defrag that comes with XP and it was
able to only reduce file fragmentation to 60%.

So I downloaded a few 3rd defraggers, eg, contig, auslogics' defragger
and dirMS. I moved all but 10GB of data to a 3rd drive, then ran the
defraggers against this remaining 10GB of files. The former
(auslogics) was able to improve fragmentation to about 33%, and the
later (dirMS) was able to get fragmentation down to under 1% and
relocate all but one file to the start of the disk.

Then I copied about 100GB of the offloaded data back on to the drive.
I ran a defrag analyze-only and noticed that the newly copied data,
although not highly fragmented, was not compacted at all - the files
were scattered all over the drive, leaving little room for new files
to be copied in without being fragmented.

Q.1)

XP appears to chop up those contiguous regions in an almost demonic
manner - determined to create the most fragments possible out of your
hard drive.

Why does XP do such a poor job selecting locations for new files as
they are copied into a hard drive with vast quantities of contiguous
free space?

Q.2)

I next ran dirMS on the drive in this state and it was not able to
compact the files together very well. I seem to be left with a drive
with plenty of space remaining, but divided amongst hundreds (maybe
thousands) of small contiguous regions which prevent new files of
modest size to be created without fragmentation right from the get-go.

Is there a good 3rd party tool for compacting files to the start of
the drive and, thus, maximizing the size of contiguous free space
regions and minimizing the number of these regions?

Thanks,
-bob
Andover, MA

 

[Leonard Grey]

Your questions seem to be based on the assumption that "...compacting
files to the start of the drive and, thus, maximizing the size of
contiguous free space regions and minimizing the number of these
regions" is necessary for good performance. But that assumption is not
correct. There is a limit to defragmentation beyond which appreciable
performance improvements are immaterial.

The amount of time you have already spent to download several
third-party programs, install them, run them, compare them and then
uninstall them has already far exceeded any performance improvement you
can expect to notice over the lifetime of your computer.

I'm not saying defragmentation is not important. What I am saying is
that unless you are operating a heavily accessed network server or some
other specialized application, the partial defragmentation and
pre-fetching performed by Windows, unglamorous as it may be, is all you
are likely to need.

 

[tsreyb]

OK on the performance points you make.

What about reliability? Doesn't file compaction equate to fewer disk
seeks over time and, hence, potentially reduced wear and tear on the
drive? If so, that wouldn't imply better long term reliability?

After all, if I drive on a 6 lane highway from Massachusetts to
California, and I switch lanes every 1/4 mile the entire way, I'd be
introducing added (and unnecessary) wear and tear on my car's
steering, suspension and tires. Why expose it to such a work load when
you don't have to?

Thanks,
-bb

 

[Gerry]

The problem you describe is fragmented free space. You can purchase
third party Disk Defragmenters that defragment free space. An example
is PerfectDisk.
http://en.wikipedia.org/wiki/PerfectDisk

You do not mention how your volumes are formatted. I will assume as
NTFS. Disk Defragmenters work in different ways so using more than one
can be counter productive. The degree of fragmentation you are reporting
is exceptional and this level normally occurs when defragmentation has
not been carried out regularly. It also will occur when there is limited
free disk space. In general terms a minimum of 15% free disk space for
the MS Disk Defragmenter to work effectively and 20 to 25% is desirable.
You can buy third party defragmenters that require negligible free disk
space to operate but most require a similar amount to the MS Disk
Defragmenter.

Before running the MS Disk Defragmenter it is desirable to run Disk
CleanUp (or a third party utility cCleaner) to empty the Recycle Bin and
remove Temporary Files etc. If you use Outlook Express you should
compact before running Disk CleanUp. Whether you use Disk CleanUp or
cCleaner you also should remove all but the latest System Restore
point -the More Options tab in Disk CleanUp. If you did not do these
tasks before running the MS Disk Defragmenter then this will partly
explain the poor result you achieved.

Some files are not movable and in the graphical display provided by the
MS Disk Defragmenter these are displayed in green. The most important of
these files is the pagefile. If this file is not contiguous then it
fragments the free disk space. This in turn fragments files being
written to disk, especially where the file is any size. To get a
contiguous pagefile after it has become non-contiguous on a disk having
less than something like 60% free space can be difficult to achieve.
Please note there is a difference between a fragmented pagefile and a
non-contuous pagefile. Some users see defragmenting a pagefile as
desirable but the reality is that it is a waste of time. The simplest
solution to the pagefile problem is to place the pagefile in it's own
dedicated partition.

My reaction to having read of your experience is that you are not
carrying out housekeeping in a systematic way. You would get better
results if you did. The two justified criticisms of the MS Disk
Defragmenter ( it is actually a cut down version of a third party disk
defragmenter) are that you cannot defragment free space and you cannot
automate the process easily. It can be made to work effectively.

If you would like comments on your current fragmentation open Disk
Defragmenter and click on Analyse. Select View Report and click on Save
As and Save. Now find VolumeC.txt in your My Documents Folder and post a
copy.

http://en.wikipedia.org/wiki/Defragment


--



Hope this helps.

Gerry
~~~~
FCA
Stourport, England
Enquire, plan and execute
~~~~~~~~~~~~~~~~~~~

 

[Leonard Grey]

You are correct: Defragmenting does save wear and tear on the read/write
heads of your hard disk. But all the moving around that occurs during a
typical defrag causes wear on the read/write heads. And it's only a
matter of time before your disk will become defragmented all over again.

Defragmenting will always be an important step, at least with the
current disk technology. My argument is that, given the increased
robustness of current file systems and operating systems, there is a
limit beyond which extra defragmentation doesn't really matter for most
uses.

This next part is only a personal anecdote, so take it for whatever it's
worth: I've had my Windows XP computer since 2002, and the only times I
have ever run XP's defragmenter manually were after I installed SP 1 and
SP 2. Earlier this year (2007) I downloaded the trial version of
Diskeeper, just to see what it would say about my hard disk. It said my
disk didn't need to be defragmented.

This last part is my personal opinion, so, again, for what it's worth: I
try to minimize what runs in the background on my computer. The popular
third-party defraggers run in the background, so why take up CPU cycles
for something that doesn't make an appreciable difference anyway?

 

[Jim]

OK on the performance points you make.

What about reliability? Doesn't file compaction equate to fewer disk
seeks over time and, hence, potentially reduced wear and tear on the
drive? If so, that wouldn't imply better long term reliability?

After all, if I drive on a 6 lane highway from Massachusetts to
California, and I switch lanes every 1/4 mile the entire way, I'd be
introducing added (and unnecessary) wear and tear on my car's
steering, suspension and tires. Why expose it to such a work load when
you don't have to?

Thanks,
-bb

Unlike automobile tires, the heads should never contact the disk surface.
If they do while the disk is rotating, the drive is toast.

Compaction results in fewer seeks. Usually, though seek time is quite short
compared to read/write time.

Compaction can make a significant reduction in total access time for large
files which are read or written from start to the end. For files which are
only accessed in pieces, such as the pagefile, compaction may not help at
all. And then there are those files which by design must be fragmented
(isn't the MFT one such file?).

Jim

 

[sareth]

Only my opinion, so take it FWIW.

Free space consolidation is not totally useless, but IMO it is a lot of
system work resulting in limited benefit except in some specific
circumstances.

Defragmenting files, especially those are frequently accessed is the
most important requirement of defragging. Doing so in the least amount
of time using the least amount of manpower and system resources is the
second consideration.

There is a point beyond which defragmenting files /free space is a net
loss in terms of time and computing resources compared to improvement
in file access times. For eg; if you have two 800MB video files,
fragmented in 2 pieces each, with a fragment of one in between the
other, there is bound to be NO extra performance in defragmenting them,
when compared to the work the drive has to do to shuffle 1.6 Gigs
around. But if those same files were fragmented into 15,000 pieces, I'd
guess that it is worth defragmenting down to the point where the files
are in a single block or just a few fragments.

Smart automatic defragmenters that can analyze the drives and
defragment when required only to the point where there is a positive
ratio of performance to resources used, are useful in work/production
environments where time = money.

As for the MFT, it has a preallocated zone (12% of the drive if I am
not wrong) so it can grow, but it will begin to get fragmented once the
drive starts to fill up and files encroach into it's zone. It will have
to be defragmented then.

 

[Gerry]

Sareth

Another thought FWIW

time = money. If the user gets on with non-computer related tasks rather
than sits and watchs progress your time = money argument falls by the
wayside.


--
Regards.

Gerry
~~~~
FCA
Stourport, England
Enquire, plan and execute
~~~~~~~~~~~~~~~~~~~

 

[sareth]

Very valid point for home users.

But I was referring to a situation where the server/workstation wa
unavailable for use because it was under system maintenance (defrag i
this case). No guarantee that other users *will* have something to d
that does not involve access to their computing resources.