ntfs v fat32

  • Thread starter Thread starter Bill Cunningham
  • Start date Start date
B

Bill Cunningham

Can someone explain this to me. Why di it take 16 min for me to copy
data from an over 80 some percent defragmented ntfs filesystem to a fat32
usb. And only 6 min to a newly formatted fat32 HD? From a fat32 usb. fat32
doesn't have large file support. That's its' onlt downside I see.

Bill
 
Bill Cunningham said:
Can someone explain this to me. Why di it take 16 min for me to copy
data from an over 80 some percent defragmented ntfs filesystem to a fat32
usb. And only 6 min to a newly formatted fat32 HD? From a fat32 usb. fat32
doesn't have large file support. That's its' onlt downside I see.

Bill
USB sticks are often slower to write to than to read from - could that
be it?
--
J. P. Gilliver. UMRA: 1960/<1985 MB++G()AL-IS-Ch++(p)Ar@T+H+Sh0!:`)DNAf

Last week, face cream. This week, the search for life on Mars. Never let it be
said /Horizon/ doesn't probe the frontiers of sciemce. - David Butcher, Radio
Times 28 July-3 August 2012.
 
Bill said:
Can someone explain this to me. Why di it take 16 min for me to copy
data from an over 80 some percent defragmented ntfs filesystem to a fat32
usb. And only 6 min to a newly formatted fat32 HD? From a fat32 usb. fat32
doesn't have large file support. That's its' onlt downside I see.

Bill

My Sandisk Ultra (joke!) Flash key, has a write speed of
3MB/sec.

If I connect one of my large backup drives to a SATA port,
the speed is 135MB/sec. Or roughly 40x faster.

That's bound to make a difference to copy time,
between those two quite different storage devices.

*******

If you can afford it, try the following hardware purchases.

1) USB3 card. They even make some (if you can still find them)
that plug into a PCI slot. For such a PCI slot USB3 card,
the transfer rate limit is 110MB/sec. For a USB3 card which
plugs into a PCI Express slot, you can easily manage 200MB/sec
on an average USB3 storage enclosure.

2) Something like a Patriot USB3 key. The dimensions of those
are going to be a bit larger than a regular flash key. Some
of the USB3 keys have multiple storage channels (the first ones
of that type had four channels). This allows it to go four times
faster than the cheapest USB3 you can find.

My $20 USB3 key does 40MB/sec. The multi-channel (fat) USB keys
can do much better than that.

So if you fork out some Benjamins, you can get better write
speed than the 3MB/sec my Sandisk Ultra manages. Check the
reviews on Newegg, where people actually benchmark their
Patriot stick, to verify the real performance numbers.
Never trust the manufacturer to tell the truth about
a product.

And another thing. If buying USB3 keys, *do not* buy a
key with a plastic barrel on the business end. USB keys
should use square metal barrels where the electrical
connections reside. The usage of stout metal, prevents
the barrel from being distorted during insertion. If
the barrel bends while you insert the key, some of the
pins can bust off. That happened to me, and I wasn't
even being rough with my new (plastic) purchase.

Paul
 
Can someone explain this to me. Why di it take 16 min for me to copy
data from an over 80 some percent defragmented ntfs filesystem to a fat32
usb. And only 6 min to a newly formatted fat32 HD? From a fat32 usb. fat32
doesn't have large file support. That's its' onlt downside I see.

Bill



Nothing to do with the file system


writing to USB devices can be slow.


I have some USB sticks that are much slower to write to than HD's


also ran into a new one yesterday with a USB self-powered HD:


The USB output on the machine I was using must have been underpowered...
as the drive seemed to hesitate a bit and copying was impossibly slow

tried the same drive on another machine and it worked perfectly


any way as I mentioned to you before...

NTFS has superior fault tolerance capabilities and if you keep using
Fat32 one of these days it's going to bite you in the butt
 
NTFS has superior fault tolerance capabilities and if you keep using Fat32
one of these days it's going to bite you in the butt

I have been using ntfs. I fragments faster but defrags faster too than
fat32. I remeber win98 and it ran on fat 32 I didn't have much trouble with
it. Fat12 was terrible. Fat 16 was so so. I remember DOS and the fat there.
Whew always getting garbage. I notice too that for some reason when resizing
a partition or splitting it. The MFT is damageed and has to be repaired. No
big deal but it's just something I noticed.

The thing about the old file systems too is that when you erase or
encrypt and shred a file it's gone. More so than on ntfs and similar modern
filesystems. I don't have anything illegal but with journal(s) files can be
reproduced that have been deleted. Things aren't so easy anymore to get rid
of. So there's two sides to that fault tolerance.

I also can't seem to find where the journal is on ntfs. So it can be
zero'd out or defragged. There's a journal inode(s) in there somewhere. If
ntfs uses inodes. Like the ext's family of filesystems. I know the
pagefile.sys that's constantly in use isn't defragged. Just complaints and
pros and cons. But alas, over 1 GB in size file is over fat32's head.

Bill
 
Bill said:
Can someone explain this to me. Why di it take 16 min for me to copy
data from an over 80 some percent defragmented ntfs filesystem to a
fat32 usb.

So still 20% fragmented. No info if the files you copied were those
that were still fragmented.
And only 6 min to a newly formatted fat32 HD?

So this HDD is 0% fragmented. And perhaps a different hard disk, too?
From a fat32 usb. fat32 doesn't have large file support. That's its'
onlt downside I see.

And why USB drives come pre-formatted using FAT32 since it has support
more places than does NTFS.

As well as using a different HDD as the source from which you copied,
did you also use different USB drives?
 
VanguardLH said:
So still 20% fragmented. No info if the files you copied were those
that were still fragmented.


So this HDD is 0% fragmented. And perhaps a different hard disk, too?

No same ntfs formatted HD. Only repartitioned and formatted with fat32
And why USB drives come pre-formatted using FAT32 since it has support
more places than does NTFS.

As well as using a different HDD as the source from which you copied,
did you also use different USB drives?

No I did not.
 
My Sandisk Ultra (joke!) Flash key, has a write speed of
3MB/sec.

If I connect one of my large backup drives to a SATA port,
the speed is 135MB/sec. Or roughly 40x faster.

That's bound to make a difference to copy time,
between those two quite different storage devices.

*******

If you can afford it, try the following hardware purchases.

1) USB3 card. They even make some (if you can still find them)
that plug into a PCI slot. For such a PCI slot USB3 card,
the transfer rate limit is 110MB/sec. For a USB3 card which
plugs into a PCI Express slot, you can easily manage 200MB/sec
on an average USB3 storage enclosure.

I am getting around 400 Mb/sec with my USB 3.0 card.

Makes image backups so much faster. :-)

Andy
 
So you used the same HDD in both tests and the same USB FAT32-formatted
drive. The only change was reformatting the HDD from NTFS to FAT32.

Was the reformatting on a data-only partition or for the OS partition?
That is, was the partition from where you copied the files also the same
partition where the OS and apps reside? Or was this for another
partition on the same HDD as the OS partition but that other partition
is just for data files. Or was the reformatted HDD a second HDD where
its partitions are just used to hold data files?

Was there a reason why you didn't defrag the other 20% of the HDD? I
ask because what if the files you copied the first time with the still
fragmented HDD were the files that were fragmented?

Was the USB drive formatted before each test? Not a quick format but a
full format. I'm wondering if each test was started under the same
condition: same set of files in an NTFS or FAT32 partition on the same
HDD and same starting state of the USB drive.

NTFS is a more robust (fuller) file system. It has journaling to
protect files during a crash, alternate data streams (data is usually in
the primary stream but a file can have more than one stream), small
files (under 4KB) can be stored immediately within the NTFS file system
since the record for the file is that big so it is easier to have the
small file right there in the file system and also no loss due to slack
space rather than having the file system pointing to an external file,
it has entries for permissions (FAT32 doesn't) which have to get
stripped when copying a file from NTFS to FAT32, supports larger volumes
(partitions) and not just larger files, however, efficiency is less with
NTFS depending on file sizes (which weren't mentioned). See:

http://www.ntfs.com/ntfs_vs_fat.htm
(under the Performance row in the table)

Also remember that the size of clusters (the size for allocation of
files within either NTFS or FAT) are different. NTFS uses 4 KB cluster
sizes whereas FAT32 can use cluster sizes ranging from 512 bytes to 16
KB depending on the partition size (which wasn't mentioned).

FAT32 is typically faster but depends on partition size, slack space,
and fragmentation. NTFS is safer along with both bigger partition and
file size support.

https://technet.microsoft.com/en-us/library/cc938440.aspx

So are you really considering using FAT32 for the partition on your HDD?
That means no data integrity and recoverability after a hard shutdown
(crash) of the OS, no file permissions so no protection on the system or
your files, smaller partition sizes, smaller file sizes, less efficiency
in storing the files on the HDD, no alternate data streams (of rare
concern to users but many programs use it), no reparse or junction
points, and probably other features that I don't recall just now.

Is copying a mass number of files to a FAT32 target something you will
do often? Will you sit idle while waiting for the copy operation to
complete so the difference in copy time is actually experienced by you?
 
I have been using ntfs. I fragments faster but defrags faster too than
fat32. I remeber win98 and it ran on fat 32 I didn't have much trouble with
it. Fat12 was terrible. Fat 16 was so so. I remember DOS and the fat there.
Whew always getting garbage. I notice too that for some reason when resizing
a partition or splitting it. The MFT is damageed and has to be repaired. No
big deal but it's just something I noticed.

The thing about the old file systems too is that when you erase or
encrypt and shred a file it's gone. More so than on ntfs and similar modern
filesystems. I don't have anything illegal but with journal(s) files can be
reproduced that have been deleted. Things aren't so easy anymore to get rid
of. So there's two sides to that fault tolerance.

I also can't seem to find where the journal is on ntfs. So it can be
zero'd out or defragged. There's a journal inode(s) in there somewhere. If
ntfs uses inodes. Like the ext's family of filesystems. I know the
pagefile.sys that's constantly in use isn't defragged. Just complaints and
pros and cons. But alas, over 1 GB in size file is over fat32's head.

Bill




Not true


the file size limit for Fat32 is 4 gigs
 
Not true


the file size limit for Fat32 is 4 gigs

OK my mistake. I have no file that size right now. I know NTFS is the
future. Other newer filesystems are all going in the same direction
including xfs which I've never used but heard good things about. I hope the
old FATs don't go away completely.

Bill
 
VanguardLH said:
So you used the same HDD in both tests and the same USB FAT32-formatted
drive. The only change was reformatting the HDD from NTFS to FAT32.

Was the reformatting on a data-only partition or for the OS partition?
That is, was the partition from where you copied the files also the same
partition where the OS and apps reside? Or was this for another
partition on the same HDD as the OS partition but that other partition
is just for data files. Or was the reformatted HDD a second HDD where
its partitions are just used to hold data files?

My HD was in 3 partions the main was data and XP on ntfs. Then a 30 GB
blank FAT32. Then 30 GB ext3. everythig was eliminated as I zero'd out the
mbr. No patition at all. I do this by linux or a mbr utility in windows I
have. I alawys defrag quick. There's no real sense to me waiting all that
time to just zero out a partion to reformat it. I just skip that and
reformat. I have a defrag utility and the defragging of the ntfs is
performed much more quickly on the ntfs filesystem. I actually like both
filesystems as far as that goes and I am weighing them together. pros and
cons. Ntfs is the future, but I don't want to see the fats go. Even Fat12. I
remember sitting in front of win98 and the power going out. With the file
sytem in memory garbage was written to disk. I turned it back on and chkdsk
ran any everything for the most part was repaired. A few found.000 files
here and there. Some truncation I'm sure. But most of the time fine. There
is exfat but it's not fully itegrated with an old XP. Otherwise there would
be no questions. Defragging fat32 is much slower than ntfs. I quick defrag.
Then do a full defrag. Ntfs fragments faster when used in conjunction with
the extended family of file systems. Much faster than with fat32.

Was there a reason why you didn't defrag the other 20% of the HDD? I
ask because what if the files you copied the first time with the still
fragmented HDD were the files that were fragmented?

I just didn't get around to it.
Was the USB drive formatted before each test? Not a quick format but a
full format.

No.

I'm wondering if each test was started under the same
condition: same set of files in an NTFS or FAT32 partition on the same
HDD and same starting state of the USB drive.

NTFS is a more robust (fuller) file system. It has journaling to
protect files during a crash, alternate data streams (data is usually in
the primary stream but a file can have more than one stream), small
files (under 4KB) can be stored immediately within the NTFS file system
since the record for the file is that big so it is easier to have the
small file right there in the file system and also no loss due to slack
space rather than having the file system pointing to an external file,
it has entries for permissions (FAT32 doesn't) which have to get
stripped when copying a file from NTFS to FAT32, supports larger volumes
(partitions) and not just larger files, however, efficiency is less with
NTFS depending on file sizes (which weren't mentioned). See:

http://www.ntfs.com/ntfs_vs_fat.htm
(under the Performance row in the table)

Also remember that the size of clusters (the size for allocation of
files within either NTFS or FAT) are different. NTFS uses 4 KB cluster
sizes whereas FAT32 can use cluster sizes ranging from 512 bytes to 16
KB depending on the partition size (which wasn't mentioned).

FAT32 is typically faster

That means a lot.

but depends on partition size, slack space,
and fragmentation. NTFS is safer along with both bigger partition and
file size support.

https://technet.microsoft.com/en-us/library/cc938440.aspx

So are you really considering using FAT32 for the partition on your HDD?
That means no data integrity and recoverability after a hard shutdown
(crash) of the OS, no file permissions so no protection on the system or
your files, smaller partition sizes, smaller file sizes, less efficiency
in storing the files on the HDD, no alternate data streams (of rare
concern to users but many programs use it), no reparse or junction
points, and probably other features that I don't recall just now.

Is copying a mass number of files to a FAT32 target something you will
do often? Will you sit idle while waiting for the copy operation to
complete so the difference in copy time is actually experienced by you?

Yes

Also forensically speaking. And security speaking. I think eliminating a
file on fat32 would be better done than with ntfs. The journal keeps a copy
of the file I would think after its deleted with ntfs. On the fats, a bit is
changed and everything is eventually rewritten over. I don't no with ntfs.
With the MFT and mirror MFT files may never completely go away. I don't have
anything illegal on my file system that I'm aware of but When I shred a file
I want it gone not hanging around in a journal. So the "safety" features of
ntfs work both ways. Sure they help protect data I understand. But they IMO
sound like they (journals) sound like they keep things hanging around too.

Bill
 
There are BIOS extensions that let you run huge drives in FAT32. I had
a 250g running for quite a while on a W/98 machine before I took the
XP plunge.
I still keep at least one FAT partition on my main machine, just so I
can still run DOS or W/98 and have an easy transport media.
It is going to be bootable.

What are the ints and function numbers? Is this 16 bit RM?

Bill
 
OK my mistake. I have no file that size right now. I know NTFS is the
future. Other newer filesystems are all going in the same direction
including xfs which I've never used but heard good things about. I hope the
old FATs don't go away completely.

Bill


At one time when I hear about the 4 gig limit I laughed and though "no
big deal"
but then realized movies can easily go over that.


I do a lot of computer repair work and data recovery
and thanks to journaled file systems such as HFS+ and NTFS
have generally been able to save most of the data on defective hard drives.
 
There are BIOS extensions that let you run huge drives in FAT32. I had
a 250g running for quite a while on a W/98 machine before I took the
XP plunge.
I still keep at least one FAT partition on my main machine, just so I
can still run DOS or W/98 and have an easy transport media.
It is going to be bootable.



I find FAT32 still useful as it's nice to have one drive where I can
easily transfer files to and from, Windows. OSX and Linux
 
Bill said:
VanguardLH wrote


Yes

Then you might want to look at repartitioning the HDD on which is the
partition containing the OS (if you only have 1 HDD). Resize (make
smaller) the OS partition leaving enough free disk space to create a new
partition that you format at FAT32. Use the FAT32 to store your big
collection of files until you get around to copying/moving them to the
USB FAT32 drive.

Note: If you move instead of copy, you might want to consider disabling
the Recycle Bin on the drive for the FAT32 partition. Moving incurs
deletion, and deletion is performed by checking if the Recycle Bin has
enough space to move a "deleted" file to that holding area. Tracking
the deletes takes time. Also, if there isn't enough room in the Recycle
Bin to delete 1 file then the latest file gets deleted from the Recycle
Bin, it checks if there is enough room for the targeted file to delete,
if not then it deletes the next latest file, and so on until there is
enough room to move the deleted file into the Recycle Bin. If the next
file to delete is huge, this same process repeats. If you delete
hundreds or thousands of files at a time, or you move that many files
which incurs the deletes, then you'll notice a big change in time to
complete a huge delete or move job with the Recycle Bin disabled on that
drive. Of course, that means losing the safety net. However, if you're
deleting or moving hundreds of huge files, not many are retained in the
Recycle Bin at any moment since they have to get deleted from there to
make room for the next huge file.
Also forensically speaking. And security speaking. I think eliminating
a file on fat32 would be better done than with ntfs. The journal
keeps a copy of the file I would think after its deleted with ntfs.

Not that I am aware of. Deleting a file in either file system merely
unallocates clusters that were previously assigned to a file. If the
file was small enough (a bit under 4 KB) to fit into the MFT allocation
unit then the file is stored there. Deleting that file merely makes
that MFT record available to another file sometime later but the file
within is not altered until that MFT record gets reused. On FAT, a
secure delete merely had to look at the chain of file table records for
the clusters containing the file and do its erasing on those clusters.
In NTFS, the same happens but the utility would also have to know about
MFT-contained files to erase those, too.
On the fats, a bit is changed and everything is eventually rewritten
over. I don't no with ntfs. With the MFT and mirror MFT files may
never completely go away. I don't have anything illegal on my file
system that I'm aware of but When I shred a file I want it gone not
hanging around in a journal. So the "safety" features of ntfs work
both ways. Sure they help protect data I understand. But they IMO
sound like they (journals) sound like they keep things hanging around
too.

Unless the secure delete utility says it will inspect the journaling
sectors, those won't get wiped. Secure deleting a file means multiple
wipes on the clusters that are/were assigned to it. I don't know if any
journaling sectors holding bits of a file are part of the MFT record for
the file. I never delved that deep into NTFS. There is the included
'fsutil' console-mode program that can manipulate the journal; however,
my only use of fsutil regarding journaling in NTFS was to turn it off
and back on but I don't remember if that was to help with defrag or to
wipe the journal's sectors. Since this is software, I figured other
utilities could also manipulate the journal, and found:

http://www.mydefrag.com/VolumeActions-DeleteJournal.html

The journal sectors on the HDD do not contain a file, per se. They
retain deltas: changes to files. I don't know that deleting the journal
(and recreate it) will actually erase anything that was previously help
in any sector previously used by the journal (and creating a new journel
doesn't necessarily reuse the same sectors as before). So I suspect you
not only need to securely delete a file but occasionally you may want to
securely wipe the free space on the drive. Before I did anything that
major, I'd save a backup (after the [secure] file deletes) to make sure
the cleanup tool didn't go amok and erase sectors it shouldn't have.

As noted at https://technet.microsoft.com/en-us/library/cc976808.aspx,
those deltas are a persistent log of changes but once the data within
each delta has been written to the file than that same journaling sector
gets reused for other changing files. If forensics are used, I suspect
the hacker would go after your pagefile before trying to use a disk
editor to find what had been left in the sectors for NTFS journaling.
Journaling sectors get reused a lot. Alas, they often get scattered in
bunches around the HDD's platters for a partition. This means you may
find that you can never a really huge file because there isn't enough
space between the journaling sectors on the HDD that would accomodate
all of the huge file. I ran into that, used some utility to tell me
where they were, and figured there was not enough space between them for
the really huge file that I was trying to defragment. The only way to
defrag the huge file into a single segment was to copy it elsewhere,
reformat the partition (which meant restoring the OS from backups),
defragment, and then copy the huge file back to that partition. Yeah,
it worked but really it wasn't worth it. I was just determined, or
maybe it was obstinence, to see if it could be done.

Because forensics usually goes after the pagefile to pick up pieces of
files (and also the memory to find data from there), some folks
configure Windows to zero out the pagefile on shutdown of Windows. This
isn't as secure as, say, using an eraser utility that makes multiple
passes on the same cluster. Also, the zeroing out of the pagefile's
sectors takes time so the shutdown takes a lot longer. Deleting files
in the file system does not affect the partial bits of a file that are
in the pagefile. Only the cluster getting reused (overwritten) by other
data would wipe out the old data there.

More info on NTFS is here: http://www.ntfs.com/. It doesn't have
everything about NTFS but it's a good starting point. I'm sure there
are some paranoid, er, security conscience forums or newsgroups out
there where their inhabitants know a lot more about ensuring your data
remains secure and and you delete isn't in memory, the pagefile, a
portion in any journaling sector, not in the HDD's cache, and if using a
secure delete utility is sufficient (and which ones cover all those
areas) or if you really need to get a HDD and software tool that support
the Secure Erase feature in some HDDs.

I only vaguely remember looking at some secure erase software
(http://cmrr.ucsd.edu/people/Hughes/secure-erase.html, freeware, and
http://partedmagic.com/secure-erase/, payware) that was used to tell a
Smart Erase capable disk to do its secure erasing. Not all HDDs support
Smart Erase but I think all SSDs do.

My brain hurts. Time for a smoothie.
 
I think I need to get to know NTFS a little bit better. The speed factor
with fat32 is definately a plus. But it's a bear to defragment, once it
indeed becomes fragmented. Security is a concern with my and NTFS. Like I
say, wiping a file I don't want things being dug back up and recreated. The
pagefile.sys doesn't really need to be used. But I have it around. If I
understood more the internals of NTFS and how secure deleted files are. More
than www.ntfs.com details deleted files I'd feel better about it.
Defragmenting ntfs is certainly a plus imo for it.

Bill
 
I cannot find fsutil anywhere. I think MS must've bought out the company
and got rid of it or something. It's mentioned on MS's site but I don't see
anywhere to download it.

Bill
 
Back
Top