Why does copying files to a new hard drive not defragment it?

[Yousuf Khan]

Well, I execute a series disk upgrades on my system yesterday and today.
I copied the contents of two drives to two other new drives. Each new
drive was obviously empty and freshly formatted beforehand. My
assumption was that copying the entire contents of each older drive
would rearrange them sequentially on the newer drives, but after several
hours of copying I found that neither of them were defragmented, and one
of them was actually more fragmented than they were in the older drive!

Looking at the map of files from Windows' defrag, it showed huge gaps of
free space scattered all over the place on the new drive, prior to
defrag. And thousands of fragmented files too. What reason would there
be for a bog standard Windows-based copy utility to not use the free
space sequentially, filling up the whole front of the drive? I assume
it's not the copy utility choosing the sections to put the files, but
probably some OS service that chooses where to locate the files?

Just annoyed, since I spent over 12 hours copying these files. I
would've expected them to be defragmented after I finished, but now I
have to go through a separate defrag step too.

Yousuf Khan

 

[Eric Gisin]

Depends what you use to copy: xcopy vs explorer, XP vs Vista, FAT32 vs NTFS.
I know when I tested it long ago in Win2K, files were mostly sequential.
(because the file copy API preallocates the destination)

 

[Rod Speed]

Well, I execute a series disk upgrades on my system yesterday and
today. I copied the contents of two drives to two other new drives.
Each new drive was obviously empty and freshly formatted beforehand.
My assumption was that copying the entire contents of each older drive
would rearrange them sequentially on the newer drives, but after several hours of copying I found that neither of them
were defragmented, and one of them was actually more fragmented than they were in the older drive!

You arent actually talking about real fragments, just fragmented free space, a different matter entirely.

Looking at the map of files from Windows' defrag, it showed huge gaps
of free space scattered all over the place on the new drive, prior to defrag.

Thats not fragmentation.

And thousands of fragmented files too. What reason would there
be for a bog standard Windows-based copy utility to not use the free space sequentially, filling up the whole front of
the drive?

The copy utility must be doing something really stupid when copying files.

I assume it's not the copy utility choosing the sections to put the files, but probably some OS service that chooses
where to locate the files?

Cant be that, because using Explorer or xcopy doesnt produce that result.

Just annoyed, since I spent over 12 hours copying these files. I would've expected them to be defragmented after I
finished, but now I have to go through a separate defrag step too.

No you dont, modern hard drives are so fast that you wont even be able to
notice the effect of a defrag in a proper double blind trial with most file ops.

 

[YKhan]

That's strange. Copying files is usually a quick way of defragmenting
them. What method did you use to copy the files?  Are the drives
system drives or data drives?  If  I do something like this I always
turn off system restore first, as it will reserve a large blank space
for restore points. Utilities such as xcopy or xxcopy will copy files
contiguously and sequentially.

Now that is interesting to hear, system restore reserves a large blank
space? I didn't realize that. Does it reserve it all in one free
space, or does it reserve free spaces all over the place?

I was using two different utilities to do the copy in each case. In
the first case, I was using a file copy util called, TeraCopy, which
is sort of a graphical xcopy, which has an elapsed time counter and
can calculate the data transfer rate at the end. I don't think it's
doing anything different than any other copy utility on the planet. I
used TeraCopy to copy off one of my data drives.

In the second case, I used XXClone which is a based on the
aforementioned XXCopy. I used this util because I was transferring my
system boot disk to a new one. XXClone can transfer a boot disk and
keep it bootable in the new drive. Again, I don't think it does
anything different to any other copy utility on the planet.

In the past I've seen this happen with good ole Xcopy as well. But
I've never seen this level of fragmentation. But when I used Xcopy
back then, my drives weren't nearly as big as they are these days, so
I assume the amount of fragmentation was less because of this too.

Yousuf Khan

 

[YKhan]

Depends what you use to copy: xcopy vs explorer, XP vs Vista, FAT32 vs NTFS.
I know when I tested it long ago in Win2K, files were mostly sequential.
(because the file copy API preallocates the destination)

Well, it was XP and the source and destinations were all NTFS in all
cases.

I used a couple of different utilties, one was TeraCopy for my data
drive, and XXClone for my system drive. I don't think either utility
does anything different than standard utilities like Xcopy or
Explorer's Copy do.

Yousuf Khan

 

[YKhan]

You arent actually talking about real fragments, just fragmented free space, a different matter entirely.

Yes I am. The Windows Defrag utility reported both file fragmentation
and free space fragmentation in its analysis report. Also reported
folder fragmentation, by-the-by.

The copy utility must be doing something really stupid when copying files..

Could be, but why would it diverge from just letting Windows itself
take care of it all?

Cant be that, because using Explorer or xcopy doesnt produce that result.

In this case I didn't use Explorer or xcopy, but this is not the first
time I've noticed this. I've noticed this for years, even in the past
when I used to use just xcopy or DOS copy: the files weren't perfectly
defragmented on the destination as I would've expected.

In this case I used XXClone to copy a system disk, and TeraCopy to
copy a data disk. You know XXClone already. Meanwhile TeraCopy is only
different from regular old XP Explorer Copy in that it includes an
elapsed time counter and a transfer rate calculator.

No you dont, modern hard drives are so fast that you wont even be able to
notice the effect of a defrag in a proper double blind trial with most file ops.

I know, I just like things to be well-organized at the beginning. I
never bother to defrag again afterwards.

Yousuf Khan

 

[thedriver]

Well, I execute a series disk upgrades on my system yesterday and today.
I copied the contents of two drives to two other new drives. Each new
drive was obviously empty and freshly formatted beforehand. My
assumption was that copying the entire contents of each older drive
would rearrange them sequentially on the newer drives,

didn't we all?

 

[Rod Speed]

YKhan wrote

Could be, but why would it diverge from just letting Windows itself take care of it all?

Software authors do all sorts of terminally stupid stuff.

XXClone has a rather sophisticated way of only copying what hasnt already
been copied, maybe that produces the rather bizarre result or something.

In this case I didn't use Explorer or xcopy, but this is not the first
time I've noticed this. I've noticed this for years, even in the past
when I used to use just xcopy or DOS copy: the files weren't
perfectly defragmented on the destination as I would've expected.

I havent used such primitive ways of migrating to a larger hard drive for a long time now.

I normally use True Image now and havent even bothered to check the
fragmentation, because I realise that it doesnt matter a damn anymore.

In this case I used XXClone to copy a system disk, and TeraCopy to
copy a data disk. You know XXClone already. Meanwhile TeraCopy
is only different from regular old XP Explorer Copy in that it includes
an elapsed time counter and a transfer rate calculator.

Thats what it looks like. Doesnt mean that the author doesnt
so something rather stupid that produces that result tho.

I know, I just like things to be well-organized at the beginning.

Why ?

I never bother to defrag again afterwards.

So why bother at the beginning ?

 

[Ed Light]

I get fragmentation with xxcopy. I've assumed it's making temp files as
it goes along. Possibly Windows Explorer will do better.
--
Ed Light

Better World News TV Channel:
http://realnews.com

Bring the Troops Home:
http://bringthemhomenow.org
http://antiwar.com

Iraq Veterans Against the War:
http://ivaw.org
http://couragetoresist.org

Send spam to the FTC at
(e-mail address removed)
Thanks, robots.

 

[Arno]

Well, I execute a series disk upgrades on my system yesterday and today.
I copied the contents of two drives to two other new drives. Each new
drive was obviously empty and freshly formatted beforehand. My
assumption was that copying the entire contents of each older drive
would rearrange them sequentially on the newer drives, but after several
hours of copying I found that neither of them were defragmented, and one
of them was actually more fragmented than they were in the older drive!

Looking at the map of files from Windows' defrag, it showed huge gaps of
free space scattered all over the place on the new drive, prior to
defrag. And thousands of fragmented files too. What reason would there
be for a bog standard Windows-based copy utility to not use the free
space sequentially, filling up the whole front of the drive? I assume
it's not the copy utility choosing the sections to put the files, but
probably some OS service that chooses where to locate the files?

Just annoyed, since I spent over 12 hours copying these files. I
would've expected them to be defragmented after I finished, but now I
have to go through a separate defrag step too.

Yousuf Khan

MS filesystems suck. The fragmentation problem has long since
been solved for the rest of the world.

Arno

 

[Yousuf Khan]

MS filesystems suck. The fragmentation problem has long since
been solved for the rest of the world.

Well, I'm not sure if it's even a problem on MS NTFS filesystems either.
I don't have any clue how fragmented files get on my Linux drives, and
that's because I don't know of any file defragmentation utilities for
Linux. Maybe that's a good thing? They must all be getting fragmented,
but it doesn't make any difference to performance.

Yousuf Khan

 

[Yousuf Khan]

I get fragmentation with xxcopy. I've assumed it's making temp files as
it goes along. Possibly Windows Explorer will do better.

Ah, okay, so you've noticed it too. Perhaps what someone else suggested
here that turning system restores off will do the trick?

Yousuf Khan

 

[Rod Speed]

Yousuf Khan wrote

Ed Light wrote

Ah, okay, so you've noticed it too. Perhaps what someone else
suggested here that turning system restores off will do the trick?

Nope, it wont produce the effect you are seeing with JUST a copy of files to the new drive.

 

[Ed Light]

Ah, okay, so you've noticed it too. Perhaps what someone else suggested
here that turning system restores off will do the trick?

I have it off. I make frequent images and use the latest one if
something happens. Data is on another partition.

I really think that the culprit is temp files.


--
Ed Light

Better World News TV Channel:
http://realnews.com

Bring the Troops Home:
http://bringthemhomenow.org
http://antiwar.com

Iraq Veterans Against the War:
http://ivaw.org
http://couragetoresist.org

Send spam to the FTC at
(e-mail address removed)
Thanks, robots.

 

[Franc Zabkar]

Well, it was XP and the source and destinations were all NTFS in all
cases.

I used a couple of different utilties, one was TeraCopy for my data
drive, and XXClone for my system drive. I don't think either utility
does anything different than standard utilities like Xcopy or
Explorer's Copy do.

Yousuf Khan

XXCLONE's "Theory of Operation" page:
http://www.xxclone.com/itheory.htm

=====================================================================
When a clone operation is performed for the first time, all the files
created on the target volume will be stored in a contiguous region.
Therefore, the clone operation in full backup mode automatically
performs the so-called "de-frag" operations.

The competing products that are based on a sector-to-sector
duplication principle propagate the same degree of fragmentation found
in the source volume to the target.
=====================================================================

- Franc Zabkar

 

[Franc Zabkar]

I have it off. I make frequent images and use the latest one if
something happens. Data is on another partition.

I really think that the culprit is temp files.

Is there an unerase utility that can tell you if any files were
created and then deleted during the copy operation?

- Franc Zabkar

 

[Arno]

Well, I'm not sure if it's even a problem on MS NTFS filesystems either.
I don't have any clue how fragmented files get on my Linux drives, and
that's because I don't know of any file defragmentation utilities for
Linux. Maybe that's a good thing? They must all be getting fragmented,
but it doesn't make any difference to performance.

Linux filesystems have very very little fragmentation, similar
to other Unix filesystems. There is an ext2 defragmenter, but
there is so little need for it, nobody ever uses it.

e2fsck with option '-v' displays the degree of fragmentation.

Arno

 

[Franc Zabkar]

MS filesystems suck. The fragmentation problem has long since
been solved for the rest of the world.

Arno

Some time ago I experimented with a floppy disc file system (FAT12):
http://groups.google.com/group/comp.os.ms-windows.win95.misc/msg/8f1507a7a914d861?dmode=source

Using Win95 DOS, I copied two small files (1 sector and 2 sectors,
respectively) to a newly formatted diskette, deleted them both, and
then copied a third file (3 sectors). Even though the FAT was empty,
the third file was copied to cluster 4 rather than cluster 2 (the
first cluster in the data area), leaving the disc fragmented.

- Franc Zabkar

 

[Ed Light]

Is there an unerase utility that can tell you if any files were
created and then deleted during the copy operation?

- Franc Zabkar

Now you're past my experience level

--
Ed Light

Better World News TV Channel:
http://realnews.com

Bring the Troops Home:
http://bringthemhomenow.org
http://antiwar.com

Iraq Veterans Against the War:
http://ivaw.org
http://couragetoresist.org

Send spam to the FTC at
(e-mail address removed)
Thanks, robots.

 

[YKhan]

Linux filesystems have very  very little fragmentation, similar
to other Unix filesystems. There is an ext2 defragmenter, but
there is so little need for it, nobody ever uses it.

e2fsck with option '-v' displays the degree of fragmentation.

Plus, I don't think the fragments that Unix file systems are talking
about are the same fragments that they talk about on Windows. In Unix,
a fragment refers to any inode that contains only partial data from a
file; usually at the end of the file, where the file won't fill the
whole inode is a fragment.

Yousuf Khan