Yea, it is not a very scientific observation. Just a casual one. =)
If those were created in a "linear" way, then the process should be
fairly "smooth" from one HD to another, tho back-and-forth thrashing
would be inevitable if the destination and target were different
volumes on the same physical HD.
Nope, SATAII. I am playing with the thought of having a 6 disk SCSI striped
raid with parity setup on my next system.
I asked, because I was wondering about possible interface intelligence
that SCSI and perhaps S-ATA/NLQ are reputed to have. That may clump
any OS burps that may pop up during the copy ;-)
Is this a system with enough RAM to prevent paging?
Now that you mention it, yea it does put some stuff towards the middle.
Though really, what is the "Middle"? There isn't just *one* middle with
today's drives having multiple platters. So really, NTFS putting things in
the middle of the linear address space of a drive is not really the middle
at all. It could be putting things on the very outer edge of a platter.
I'm pretty sure linear addressing (LBA) will fill cylinders before
clicking heads forward. A "cylinder" being the logical construct of
all tracks on all platter surfaces that can be accessed without moving
the heads - that could span physical HDs in RAID 0, for example.
Exceptions might arise when HD firmware (or OS "fixing") relocates
failing sectors (or clusters) to OK spares. Shouldn't really be an
active "feature" in a healthy setup, tho.
I know Ext2/3 has duplicates of the superblock which defines the file
system itself.
NTFS duplicates the first few records of the MFT, and that's it. Not
sure if that includes the run start/length entries for all files,
which would be NTFS's equivalent to FAT1+FAT2.
It sounds as if Ext2/3 does something similar, depending on just how
far "defines the file system itself" takes you.
Yea but it doesn't seem to pre-allocate all that much.
Depends on how the NTFS was created, perhaps - i.e. formatted as NTFS,
or converted from FATxx, etc. I know there's an OEM setup parameter
to address this, which sets aside space for an eventual NTFS MFT after
the initial FATxx file system is converted later. I think this is to
cater for a "FAT32, install/image OS, convert NTFS" build method that
old build tools may require some OEMs to use.
Also, not everything that is "can't move" is MFT; some may be
pagefile, etc. Not sure how Diskkeeper shows this info...
As I've said above, it can be defragmented with the right software. But
otherwise, you probably would be stuck. =)
Yup. Also, does defragging purge old entries?
I kinda like the old FATxx design, for one particular reason; it's
easy to preserve and re-assert the file system structure in data
recover scenarios, if you know exactly where it will be.
Actually, it does pre-create all the primary structures when you first
format the disk. This does mean that Ext2/3 has a fixed limit of how many
files you can theoratically create in the file system.
Hmm... that sounds like a missed "scalability" opportunity to me.
(keeping structure metadata and file content together...)
The idea of moving around files that are frequently used is not a bad one.
Since Win98, this has changed the objectives of "defrag" from
"defragging files" to "optimizing access to frequently used code", so
apps and OS start up faster. This can have the effect of fragmenting
large files that have only some parts "frequently used", which would
be sacrilige to an "old logic" defragger.
Though I do see some problems with it:
- Need enough contiguous space to group the files together.
That's what the slow "first 10%" in a Win98 defrag is all about
- Need time to actually do so. It doesn't help any if it takes more time
to move the files around than it would take to simply read them from
opposite ends of the disk.
Defrag = "duty now for the future", where the intention is to waste
time defragging at times the user isn't trying to work. Underfootware
defragging is a Vista idea that I have yet to warm to.
What's more of a problem is the thrashing that can happen when the
status of files is percieved to change. An overly-sensitive
"frequently-used" logic could cause that, so you need a bit of
hysteriesis (spelling?) to lag that from bouncing around.
- There is no single "start" and "end" of the disk. Multiple platters and
2 sides per platter mean multiple starts and ends.
Logically, all surfaces, heads and tracks of "the disk" are considered
as one, and (hopefully) addressed in a sequence that minimizes head
travel. This logic has been present since DOS first dealt with
double-sided diskettes, where they are filled in sector, head, track
order (rather than the sector, track, head order you intuit).
My first disk system didn't have that logic; it treated each side of a
diskette as a separate disk drive! That was a home-cooked add-on for
the ZX Spectrum, which I worked with quite a bit.
- How frequently does a file need to be accessed before it is considered
frequent? What if the usage pattern changes a lot?
Yup. Hello, context thrashing.
Actually most defraggers these days don't really bother doing that
anymore all that much. Diskeeper actually has a section in its FAQ
answering why it doesn't. =)
Which means if you want to concentrate travel within a narrow band,
then leaving it to defrag isn't the hot setup - better to size
partitions and volumes and apply your own control to what goes where.
Else, as far as I can see, you will always have a shlepp from the
old/fast code at the "front" to the new stuff at the "end", The fix
for that is to suck out dead stuff from the "middle" and stash it on
another volume, then kick the OS's ass until it stops fiddling with
this inactive stuff via underfootware processes.
That includes the dead weight of installation archives, backed-up old
code replaced by patches, original pre-install wads of these patches,
etc. which MS OSs are too dumb to do. A "mature" XP installation can
have 3 dead files for every 1 live one... nowhere else would that sort
of inefficiency be tolerated.
Fair enough - that's an easy case to solve, and worth solving. I
think MS's logic is usually to create new files in the unbroken mass
of free space at the "end" of the file system, only filling up the
"lacunae" between files when the volume's wave of files hits the end
of the volume. For this reason, a volume that has run out of space is
worth defragging once you free up enough space, for the same reason
that that defrag will take a lot longer than usual
True, but this is partially mitigated by the fact that file data is kept
in memory as long as possible and committed as late as possible. So that
way, even if the context isn't aware of the file size, the file system
handler is by keeping the content in RAM if available.
Hmm... yes, I can see that can help. Means the start of the file is
only comitted when the file is flushed for the first time.
Another strategy is to work on temp files, then rebuild the "real"
file only when the file is formally saved. This is what Word does,
and why there are always those ~ ghosts lying around if Word gets
bad-exited (those may be the "real" file, due to replace the one you
see only when saved).
Well both Windows and Linux use all available cores for task scheduling.
However, as far as the application is concerned, if it is a single
threaded app it will not use more than 1 core.
Sure, fair enough. By now, one might expect apps doing "heavy things"
to spawn multiple threads, and Vista's limits on what apps can do
encourages this, i.e. splitting underfootware into parts that run as
service and as "normal app" respectively.
These days, you need a spare core just to run all the malware ;-)
An application, in any operating system, does have to be specifically
written to take advantage of multiple cores or processors in a system.
This also presents challenges from a programming point.
Yup, no lie there. I think the compiler will take care of some
details, as long as you separate threads in the first place.
AFAIK NT's been multi-core-aware for a while, at the level of separate
CPUs at least. Hence the alternate 1CPU / multi-CPU HALs, etc.
Not sure how multiple cores within the same CPU are used, though - it
may be that the CPU's internal logic can load-balance across them, and
that this can evolve as the CPUs do. I do know that multi-core CPUs
are expected to present themselves as a single CPU to processes that
count CPUs for software licensing purposes.
It's been said that Vista makes better use of multiple cores than XP,
but often in posts that compare single-core "P4" core processes with
dual-core "Core 2 Duo" processors at similar GHz, without factoring in
the latter's increased efficiency per clock.
So they may in fact only be telling us what we already know, i.e. that
the new cores deliver more computational power per GHz.
A word processor for example will probably never benefit much from
more than 1 core.
No, I can see that being an example where multiple cores would help;
background pagentaion, spell-checking, and tracking line and page
breaks, for example - the stuff that makes typing feel "sticky". Not
to mention the creation of output to be sent to the printer driver,
background saves, etc. Non-destructive in-place editing can be a
challenge, and solutions often involve a lot of background logic.
Multithreading is also not easy to deal with from a programming
point. There are a LOT of things that can go wrong, so usually a lot of
programmers avoid it. There also was very little, if any, benefit for it
in the desktop market before multi core CPUs.
Some of that challenge applies to any event-driven UI, as opposed to
procedural or "wizard" UIs. IOW, solutions to that (which unlink each
action from the base UI module, and sanity-check states between the
various methods spawned) may also pre-solve for multi-core.
I've only used multiple threads where not doing so would freeze the UI
while it waits on the task to be completed (triggered by a button press
for example).
Sure. I used to "hide" some overhead by doing some processing
straight after displaying a UI, on hte basis that the user will look
at it for a while before demanding attention (this is back in the PICK
days) but folks familiar with their apps will type-ahead and catch you
out. There are all sorts of ways to go wrong here, e.g...
- spawn a dialog with a single button on it called OK
- when presses OK, start a process
- replace that button with one to Cancel
- when the process is done, report results
- relace that button with one for OK to close the dialog
Multithreading can also sometimes introduce more overhead than it actually
gains in performance, though this is less of a problem with multi-core
CPUs than it is with a hyperthreaded P4.
I think true multicores may be more "transparent" that HyperThreading,
in that HT prolly can't do everything a "real" core can. So with HT,
only certain things can be shunted off to HT efficiently, whereas a
true multicore processor will have, er, multiple interchangeable cores
There's no doubt that "computing about computing" can pay off, even in
this age of already-complex CPUs. The Core 2 Duo's boost over
NetBurst at lower clock speeds is living proof of that, and frankly, I
was quite surprised by this. I thought that gum had had all its
flavor chewed out of it; witness the long-previous "slower but faster"
processors from Cyrix, and AMD's ghastly K5.
One easiest way to multithread an application, if applicable, is to
create "tasks" and have a master thread manage these tasks
and distribute these to worker threads to perform.
Isn't this pretty much what Windows' internal messaging stuff does? I
know this applies to inter-process comms, I just don't know whether
the various bits of an app are different processes at this level.
Perhaps some of this logic can be built into the standard code and UI
libraries that the app's code is built with?
I myself am in the boat right now. Application I am developing could in
theory benefit from multithreading but currently is single threaded and
will only use one core. The reason I don't go multithreading is because
doing so would greatly increase the complexity of my code and at this
point in time I simply don't have the need performance wise.
The problem is race conditions between the two processes. If you did
spawn one process as a "loose torpedo", the OS could pre-emptively
time-slice between it and it could come to pass that the two halves
wind up on different cores.
Even on my P4 system I can throw 300k triangles, which is a fairly complex
dataset,to be rendered at the processor and still get more than 60 frames
per second. And that's on a system that wouldn't even really benefit from
multi threading the rendering pipeline. My core 2 duo just yawns at me and
begs me to give it more data with the other core sound asleep. =)
Heh heh ;-)
I think modern games are often built on engines; specifically, an
engine to do graphics and another for sound. The AI behind the
character's activities can be split off as an asynchronous thread,
whereas sound and graphics have to catch every beat.
On top of that, my single threaded engine isn't even as fast as it could
be. I currently still store vertex data in system memory. If I actually
stored it in video memory the speeds would be even more insane than they
are.
So even for me, multithreading is somewhere at the extreme bottom of
performance enhancements that I might think about.
It may help at the OS level, e.g. when copying files, the av scanner
coukld be running in the "spare" core. I think this was a large part
of the thinking behind HT... also, there will be network packets to be
batted off by the firewall, etc. and that's a coreful of stuff, too.
Well, perhaps not a complete corefull, but YKWIM.
Real-time stuff like MP3 playback (something else that may be built
into games, with the compression reducing head travel) can also
benefit from having a spare core to process them.
And then there's all the DRM trash to compute, too...
---------- ----- ---- --- -- - - - -
Proverbs Unscrolled #37
"Build it and they will come and break it"
n a