Fragmentation of large video file

S

Stan Hilliard

OS=Windows XP Pro SP2
I downloaded a 1 hour video, file dn008.0318.mp4 from democracynow.com
using the video player Miro.exe.

The 451,294 KB, file had 6,264 fragments. When I use this process I
typically get this degree of high fragmentation.

Q1 - Is such high fragmentation to be expected with video files? Or
does this indicate that I have a problem?

Windows defrag couldn't defrag it (or several other video files.)
Piniform defraggler showed 3 fragments after Defragging it.

Information will be appreciated,
Stan Hilliard
 
A

Alec S.

Stan Hilliard wrote (in
The 451,294 KB, file had 6,264 fragments. When I use this process I
typically get this degree of high fragmentation.

Q1 - Is such high fragmentation to be expected with video files? Or
does this indicate that I have a problem?

The more you create, move, and delete, files, the more and more fragmented the
disk becomes. That’s just how filesystems work. If they were to rearrange all
files every time you performed a file operation, performance would be horrible
and the disk would wear out quickly. That’s why you’re really only supposed to
do a defrag once a week, or maybe day.

A 450MB file is pretty big and requires a whole lot of clusters. If you don’t
have enough free space, then Windows cannot find 450MB worth of consecutive free
clusters in which to place the file, and thus splits it into chunks. The defrag
apps attempt to consolidate the chunks into the fewest, largest chunks they can.

Having 6,264 fragments isn’t terrific, but it’s not horrible either. For a file
that big, on a heavily-used disk without tons of free space, it’s pretty
average. If your disk uses a typical 4KB cluster size, that video would need
112,824 clusters, which means that it has 5.55% fragmentation.

It is best to clear out as much junk as you can before defragging. Delete temp
files, browser caches, etc. to free up as much space as possible. If a row of
120,000 free clusters is broken by even a single cluster, say a tiny GIF file in
the browser cache, then your video would not fit into it in a single chunk and
would have two fragments.

It’s rare that a video is framented enough to impact the performance (ie the
video actually stalls while waiting to be read from the disk). It would have to
have each cluster in a separate fragment, spread out on the disk in an
unrealistic pattern, and the disk would have to be pretty slow. I doubt you
would notice anything while watching it.

That said, having as few fragments as possible makes data recovery much easier
(or in some case possible at all), not to mention general, overall system
performance is improved.
 
S

Stan Hilliard

Stan Hilliard wrote (in

The more you create, move, and delete, files, the more and more fragmented the
disk becomes. That’s just how filesystems work. If they were to rearrange all
files every time you performed a file operation, performance would be horrible
and the disk would wear out quickly. That’s why you’re really only supposed to
do a defrag once a week, or maybe day.

A 450MB file is pretty big and requires a whole lot of clusters. If you don’t
have enough free space, then Windows cannot find 450MB worth of consecutive free
clusters in which to place the file, and thus splits it into chunks. The defrag
apps attempt to consolidate the chunks into the fewest, largest chunks they can.

Having 6,264 fragments isn’t terrific, but it’s not horrible either. For a file
that big, on a heavily-used disk without tons of free space, it’s pretty
average. If your disk uses a typical 4KB cluster size, that video would need
112,824 clusters, which means that it has 5.55% fragmentation.

It is best to clear out as much junk as you can before defragging. Delete temp
files, browser caches, etc. to free up as much space as possible. If a row of
120,000 free clusters is broken by even a single cluster, say a tiny GIF file in
the browser cache, then your video would not fit into it in a single chunk and
would have two fragments.

It’s rare that a video is framented enough to impact the performance (ie the
video actually stalls while waiting to be read from the disk). It would have to
have each cluster in a separate fragment, spread out on the disk in an
unrealistic pattern, and the disk would have to be pretty slow. I doubt you
would notice anything while watching it.

That said, having as few fragments as possible makes data recovery much easier
(or in some case possible at all), not to mention general, overall system
performance is improved.

Thanks Alec,

Your explanation suggests to me that I should be defragging the free
space. Isn't XP-Pro smart enough to start saving at the location on
the disk with the most contiguous free space available to hold the
file?

Stan Hilliard
 
A

Alec S.

Stan Hilliard wrote (in
Your explanation suggests to me that I should be defragging the free space.

You should defrag once in a while, but don’t over-do it. If a defrag utilit
analyzes your disk and says that it doesn’t need it, don’t bother unless you
have a good reason; you won’t get any tangible benfit and will just add
unnessceary wear-and-tear.

Isn't XP-Pro smart enough to start saving at the location on the disk with
the most contiguous free space available to hold the file?

Windows uses a smart algorithm to determine where to store files. In the really
old days, DOS would just store in the next available cluster, but yes, Windows
does take into account fragmentation and tries to avoid it when possible.
 
B

Bill Sharpe

Stan said:
Thanks Alec,

Your explanation suggests to me that I should be defragging the free
space. Isn't XP-Pro smart enough to start saving at the location on
the disk with the most contiguous free space available to hold the
file?

Stan Hilliard
If the video plays without stuttering or breaking up I'd say don't worry
about the amount of fragmentation.

Bill
 
T

Twayne

OS=Windows XP Pro SP2
I downloaded a 1 hour video, file dn008.0318.mp4 from democracynow.com
using the video player Miro.exe.

The 451,294 KB, file had 6,264 fragments. When I use this process I
typically get this degree of high fragmentation.

Q1 - Is such high fragmentation to be expected with video files? Or
does this indicate that I have a problem?

Windows defrag couldn't defrag it (or several other video files.)
Piniform defraggler showed 3 fragments after Defragging it.

Information will be appreciated,
Stan Hilliard

It isn't literally the number of fragments that is a problem. More
important is where the fragments are located. And you can't tell that
from most defrag maps, BTW. If all the fragments are located in the
same general area of the disk, it's likely it'll never be noticed in any
way; the heads get to the data almost as fast as if they were
contiguous. If however the fragments are spread all over the platters
so the head has to constantly go from the inside to the outside, jump to
the middle, then back out, etc. etc. etc., you're a lot more likely to
notice the fragmentation, especially in video. Other factors of course
are the number of platters, heads and speed (rpm) of the drive, buffer
size, how it handles look-aheads, etc etc etc..
Windows takes most of that into consideration when it recommends or
not to defrag a drive. That's why occasionlly you'll see it say no need
to defrag the 7,000 fragments but another time it'll say the drive is
badly fragmented, but there will only be, say, 1,000 fragments in the
largest fragmented file. So, the quantity isn't the important number:
The work and access speed of the drive are the important numbers.

IMO defragging has no bearing on "wear and tear" on a drive; it's
irrelevant. If the MTBF on a disk is 5 years, you will not get ten
years out of it by doing exactly half of the accesses that would occur
in the five years. It just doesn't work like that.

It's worthwhile for most to pay attention to how long it takes to defrag
their drives for awhile, and then develop a schecule for it that
accomodates it. If you defrag too little, the defrags take longer to
run. If you defrag more often, they will take less time to run.
Eventually you'll reach a point where more frequent defrags don't run
any faster than the previous, so ... there abouts is your magic time
frame.
In my case that's about once a month most of the time. But, should I
happen to be doing video editing and rendering, then I defrag after
every session and before I start the next. When you get into things
where huge masses of data are being moved, created, deleted and
modified, fragmentation happens quickly. I even had a separate physical
drive for that work so that only one drive needs to be defragged and
kept track of.

That's my 2 ¢ anyway,

Twayne
 
A

Alec S.

Twayne wrote (in
IMO defragging has no bearing on "wear and tear" on a drive; it's
irrelevant. If the MTBF on a disk is 5 years, you will not get ten
years out of it by doing exactly half of the accesses that would occur
in the five years. It just doesn't work like that.

Sure it does. The more the head has to move back and forth, the quicker the
armateur will fail and the greater the chance of the head crashing. Granted,
drives today are built much better than in the past (although even that’s
debateable), but that does not change the fact that the more that phyiscal parts
move, the faster they will fail.

The M in MTBF stands for mean, in other words, an MTBF of five years means that
/on average/, it will be five years between failures, but it depends on usage,
environment, etc. If you use the crap out of the drive, it *is* usually going to
die sooner than one that’s idling most of the time. Of course it depends on the
activity and parts involved. For example, defragging alot would result in the
heads/arms dying quicker, but not the spindle, but powering up/down a lot
*would* cause the spindle to die because of the frequent spin-up/down.

Think about your body. Doing a lot of reading would wear your eyes, doing a lot
of typing/writing would wear your arms and fingers, doing a lot of weightlifting
would wear your joints and knees; and this is with organic systems that can
repair themselves!

It’s not really simple, there are confounding factors, but yes, generally
speaking, you don’t want to over-do it with defragging; at best it just wastes
electricity and creates heat without providing much in return (unless you’re
obsessive coumpulsive and need it nice and organized). If solid-state drives
could be created with something more resiliant and faster than flash-RAM, then
drives would become really amazing: very fast, cool, low-power, and reliable.
 
O

Onsokumaru

Alec S. said:
Twayne wrote (in

Sure it does. The more the head has to move back and forth, the quicker
the
armateur will fail and the greater the chance of the head crashing.
Granted,
drives today are built much better than in the past (although even that's
debateable), but that does not change the fact that the more that phyiscal
parts
move, the faster they will fail.

The M in MTBF stands for mean, in other words, an MTBF of five years means
that
/on average/, it will be five years between failures, but it depends on
usage,
environment, etc. If you use the crap out of the drive, it *is* usually
going to
die sooner than one that's idling most of the time. Of course it depends
on the
activity and parts involved. For example, defragging alot would result in
the
heads/arms dying quicker, but not the spindle, but powering up/down a lot
*would* cause the spindle to die because of the frequent spin-up/down.

Think about your body. Doing a lot of reading would wear your eyes, doing
a lot
of typing/writing would wear your arms and fingers, doing a lot of
weightlifting
would wear your joints and knees; and this is with organic systems that
can
repair themselves!

A very poor analogy - doing nothing will cause your body to atrophy, and
performance will be extremely low - far lower than remaining active.

You don't, "wear out", your eyes by reading, you can damage them by abusing
them, but not wear them out.

Do you, "not think", in order to save your brain?

MTBF isn't measured by running a drive till it dies.
http://www.storagereview.com/guide2000/ref/hdd/perf/qual/specMTBF.html

Of course using a device causes wear, and it will wear out eventually be
being used, but if the HDD is active the amount of extra time spent
defragging is probably so small it's not worth thinking about.

Of course if you set your machine to defrag every minute, then the amount of
time would be significant.

You would have to know exactly when the drive would fail to measure the
impact, but I doubt many people wait till the drive fails, then add up the
time they spent defragging and curse those lost hours.

It's more like saying you won't drive your car one extra mile a week because
over a year you will, "lose", fifty two miles, and you could have travelled
that much further before the car broke down.
 
T

Twayne

Twayne wrote (in

Sure it does. The more the head has to move back and forth, the
quicker the armateur will fail and the greater the chance of the head
crashing. Granted, drives today are built much better than in the
past (although even that’s debateable), but that does not change the
fact that the more that phyiscal parts move, the faster they will
fail.

Yes; the more physical parts move, the more they wear. But you are
assessing much more import to it in this instance than is reasonable;
it's a negligible fact within the forest of facts that lead up to a
drive failure.
If head movement due to defrag were the only thing involved, and
defrag was the only thing that made the heads move, your conjecture
might have something to it. But that is not the real world.

Again I recommend you do more research if this is an important issue
to you because it's negligible in the overall scheme of things. You are
producing seemingly logical proposals which could only be true in a
controlled, specific-activity session of the drives.
The number of "normal" (of which the variations are likely almost
infinite) head movements, reads and writes if much much higher than any
defrag run at even weekly or daily intervals.
The vast majority of the time, even when the machine is sitting
idle, the heads may be in service, doing some of the many background
tasks normally accomplished in any operating system. You would have to
create a very specific, controlled situation for your conjecture to
become more than negligible.
Using your logic, keeping a drive fully fragmented would result in
almost no wear of the heads because they would never have far to move
and fewer occasions to move.
The M in MTBF stands for mean, in other words, an MTBF of five years
means that /on average/, it will be five years between failures, but
it depends on usage, environment, etc. If you use the crap out of the
drive, it *is* usually going to die sooner than one that’s idling
most of the time. Of course it depends on the activity and parts
involved. For example, defragging alot would result in the heads/arms
dying quicker, but not the spindle, but powering up/down a lot
*would* cause the spindle to die because of the frequent
spin-up/down.

M stands for "Mean"; right. However, "Mean" does NOT mean "average".
Again, more research on your part would clarify this for you. And, once
more, if the ONLY thing going on was what you describe, your conjecture
might become more than moot. But again, I say, do some research on the
subject and become more informed on these things and you will understand
them better. Holding fast to one logical assumption you have made at
the expense of ignoring all others is not the way to figure these things
out. You need the forest, not just a couple of trees in the forest of
interest.
Think about your body. Doing a lot of reading would wear your eyes,
doing a lot of typing/writing would wear your arms and fingers, doing
a lot of weightlifting would wear your joints and knees; and this is
with organic systems that can repair themselves!

Nah, bad analogy and again, it ignores the many other things that have
an impact on such things. e.g. reading does NOT "wear" your eyes. Your
eyes are "seeing" 24/7, even while you're asleep, and muscle movement is
nearly constant. Eye movement is even noticeable to others during
certain stages of sleep. Analogies such as this go nowhere; stick to
facts, not intimations and analogous situations.
It’s not really simple, there are confounding factors, but yes,
generally speaking, you don’t want to over-do it with defragging; at
best it just wastes electricity and creates heat without providing
much in return (unless you’re obsessive coumpulsive and need it nice
and organized). If solid-state drives could be created with something
more resiliant and faster than flash-RAM, then drives would become
really amazing: very fast, cool, low-power, and reliable.

At best, defragging too much keeps defrag times short and there is
little to do each time defrag runs. Defragging a drive that has already
been properly defragged can literally take only a few seconds to
complete. I see it happen fairly often when I happen to defrag a drive
that hasn't become fragmented. So, even by your conjecture, there would
be little head movement beyond reading the tables to see what needs to
be fragmented.
If you're using a 3rd party defragger and it's taking a long time to
run, even on a properly already defragged drive, then I would suggest
you look into a better tool. Unfragmented data should NOT be moved;
ever, by a decent defrag program.
The ONLY drive that becomes fragmented during non-use of that drive
is the operating system drive. That's because it's reading and writing
almost constantly, usually to the registry but also in performing any of
the many background tasks that may be set to run while the machine is
idle. Other non-system drives will not fragment, so defragging one of
those only takes a couple tens or so of seconds, or should anyway.
There is nothing to do on an already defragged disk drive.

Once more, I'll say it: Do some research on the issues if this is
imortant to you. If it isn't, and you simply have a closed minded
objection to anyone having the audacity to call you wrong, well, you're
just showing your own ignorance and inability to refrain from giving
mis-information. Posting unverifiable mis-information is only slightly
better than being a 1. spammer and 2. a troll. But a person who will
check things out when they've ben advised their information might be
incorrect, well, is a thinking person interested in the accuracy of his
posts and the well being of those he advises.

Should I say it again? Do some research. I'm not providing URLs to
verifying this information for a couple of reasons: It's easy to find,
and I have a feeling even white papers would simply fly by you without
even a glance at them. You have more interest in being thought to be
right than you do in actually being accurate.

Unless you have something intelligent to say, I'll likely not be
responding further you in this thread. I've said what I need to say and
all I'd end up doing is repeating the same information again.

HTH

Twayne
 
A

Alec S.

Onsokumaru wrote (in
A very poor analogy - doing nothing will cause your body to atrophy, and
performance will be extremely low - far lower than remaining active.

It’s not about doing nothing, it’s about overdoing something. You can sharpen a
knife once in a while, but if you do it too much, it will wear out much faster.
You can polish jewlery once in a while, but if you do it too much, you will wear
it out. Sharpening and polishing provide a benefit when done when needed, but if
you overdo them (eg when not needed), they will wear out the product without
providing anything of value. Better analogies?

You don't, "wear out", your eyes by reading, you can damage them by abusing
them, but not wear them out.

Fine, not “wear them out”, “break”. Happy? If you try to read lots and lots of
small print, forcing your eyes to exert themselves excessively, then they will
become exhausted and the muscles will stop functioning correctly until they can
heal (they can’t contract or relax the proper amounts). If you tax the moving
parts of a hard drive, they will break, they will fail, the oil will crap out,
the head will have more opportunites to gouge into the disk, etc.

Do you, "not think", in order to save your brain?

Yes. :p
 
A

Alec S.

Twayne wrote (in
If head movement due to defrag were the only thing involved, and
defrag was the only thing that made the heads move, your conjecture
might have something to it. But that is not the real world.

??? If the sky were blue and blue were the color of the sky?

Anyway, defrag causes a lot of head movement; that’s its nature. Have you ever
watched one go? They move blocks of data from end of the drive to the other
repeatedly, many, many times. That’s a lot of head movement. The more the head
flys around, the more opportunites for disaster. (Of course that’s all the more
reason to defrag, but again moderation is the key.) I have yet to see a
defragger that can use another (physical) disk as temp storage instead of the
free space on the partition being defragged.

Again I recommend you do more research if this is an important issue
to you because it's negligible in the overall scheme of things. You are
producing seemingly logical proposals which could only be true in a
controlled, specific-activity session of the drives.

Specific activity? For example? Like I said, watch a defrag session.

Using your logic, keeping a drive fully fragmented would result in
almost no wear of the heads because they would never have far to move
and fewer occasions to move.

??? Fragmentation only has any bearing when the files are being accessed. If one
file is being accessed, then the other files have no bearing on head movement.
If the drive were fully fragmented /and/ every file on the disk were being
accessed (eg copying all files to another disk or something), then it would be
similar to a defrag session.

M stands for "Mean"; right. However, "Mean" does NOT mean "average".

Sorry, but I don’t have the educational backgrounds of everyone on the Internet.
If I knew that you had a degree in statistics, I would have been more specific.
And for the record, yes it does; a mean is one measure of “average”.

Your eyes are "seeing" 24/7, even while you're asleep, and muscle
movement is nearly constant.

Seeing and reading are not the same thing. When you read, your eye muscles
contort to a different configuration to accomodate the different focus. If you
do that too much, the muscles will exhaust. Maybe you can read thousands of
pages of tiny print or run for thousands of miles without problem, but most
people cannot; most of us have limits and if we repeatedly exceed them to the
point of injury, we run into problems, many of which have permanent effects.

If you're using a 3rd party defragger and it's taking a long time to
run, even on a properly already defragged drive, then I would suggest
you look into a better tool. Unfragmented data should NOT be moved;
ever, by a decent defrag program.

I remember in the days of DOS, defraggers would have the option to actually
reorganize the files according to directory structure (eg directory A would be
moved to the front followed by all of its subdirectores and files, then
directory B, and so on). Defraggers don’t seem to do that anymore, they only
make sure that files are in one piece (although most do try to push all files
towards the start of the disk in whatever order).

Once more, I'll say it: Do some research on the issues if this is imortant to you…
Posting unverifiable mis-information is only slightly
better than being a 1. spammer and 2. a troll.

You know, with all of your nagging, you never indicated whether you have any
education in the matter yourself. You never said that you are an engineer, or
anything; no credentials listed. All you do is say “wrong, wrong” without
correcting. That is only slightly better than being 1. a spammer and 2. a troll.

But a person who will check things out when they've ben advised their information
might be incorrect, well, is a thinking person interested in the accuracy of his
posts and the well being of those he advises.

Okay, your information is incorrect, inaccurate, and downright wrong. Go look it
up and do some research.

I'm not providing URLs…

See what I mean.

Unless you have something intelligent to say, I'll likely not be
responding further you in this thread. I've said what I need to say and
all I'd end up doing is repeating the same information again.

Thank the Hevean—and Hell.
 
P

Plato

Stan said:
OS=Windows XP Pro SP2
I downloaded a 1 hour video, file dn008.0318.mp4 from democracynow.com
using the video player Miro.exe.

The 451,294 KB, file had 6,264 fragments. When I use this process I
typically get this degree of high fragmentation.

Q1 - Is such high fragmentation to be expected with video files? Or
does this indicate that I have a problem?

Windows defrag couldn't defrag it (or several other video files.)
Piniform defraggler showed 3 fragments after Defragging it.

Perhaps your hard drive is getting full. Before downloading large files
make sure you defragged in advance so the new file is written to the
disk without fragmented.
 
A

Alec S.

Twayne wrote (in
LOL! And there, we have it in a nutshell! It's called atrophy. Use it
or lose it.

Hmmm, iteresting that you clearly have no concept of humor yet use “LOL”.
Mimicking emotions that one cannot themselves feel is classic sociopathic
behavior.
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Top