win2k/xp/2003 file I/O, buffering, queues, hdd seek

[Guest]

hi,

i am wondering for some time now if windows 2k/xp/2003 file I/O could be
optimized.

situation:
- a single I/O operation performing sequential read or write is very fast,
probably limited by harddisk read / write speed (assuming no drive
fragmentation, I/O between two different harddisks).
- as soon as concurrently more file operations are running (on the same
drive) the overall I/O performance considerably drops! heavy harddisk seeks
can be observed.

conclusion:
- windows switches (too) fast between the concurring operations which in
turn causes hdd seeks which causes performance to break in.

open questions (assuming two competing I/O operations) :
1. how large is the buffer windows uses for file operations? stated
differently: after processing how many bytes is the other operation executed?
where is it defined?
2. is it possible to change the time / bytes how often a switch between the
operations occur?
3. could a windows driver perform I/O queue reordering (like "traffic
shaping") to reduce hdd seeks? (take into account the destination partition /
drive,...)? maybe implement some optimized caching (returning "write
successfull" to the calling function but driver waits for 100 more ms for
incoming adjacent write operations and then schedule them to run
consecutively? have priorities for operations based on user account,..?

thanks, volker.

 

[Jim Lewandowski]

hi,

i am wondering for some time now if windows 2k/xp/2003 file I/O could be
optimized.

situation:
- a single I/O operation performing sequential read or write is very fast,
probably limited by harddisk read / write speed (assuming no drive
fragmentation, I/O between two different harddisks).

***
The controller will destage cyl/head/sectors to try to stay ahead of the CPU usage OF that
data.

- as soon as concurrently more file operations are running (on the same
drive) the overall I/O performance considerably drops! heavy harddisk seeks
can be observed.

conclusion:
- windows switches (too) fast between the concurring operations which in
turn causes hdd seeks which causes performance to break in.

***
Windows is likely processing the I/O operations in FIFO order. It might make sense (maybe
VISTA has it) to allow the user to control I/O processing by "sorting" as you suggest
(kind of like you can specify CPU priority based on background or foreground
tasks/programs currently).

open questions (assuming two competing I/O operations) :
1. how large is the buffer windows uses for file operations? stated
differently: after processing how many bytes is the other operation executed?
where is it defined?

***
I haven't fully figured this out as the Windows I/O requests are LOGICAL (i.e. give me 100
bytes) which in turn causes the controller to read a MINIMUM of the "cluster size" (most
people format their HDs with 4K cluster size). Use a tool like diskmon or filemon from
http://www.sysinternals.com to monitor the I/Os.

2. is it possible to change the time / bytes how often a switch between the
operations occur?

***
Likely not.

3. could a windows driver perform I/O queue reordering (like "traffic
shaping") to reduce hdd seeks? (take into account the destination partition /
drive,...)? maybe implement some optimized caching (returning "write
successfull" to the calling function but driver waits for 100 more ms for
incoming adjacent write operations and then schedule them to run
consecutively? have priorities for operations based on user account,..?

***
At a MINIMUM, in Device Manager for your hard disk(s), specify (check) the POLICIES tab
and make sure you've checked "enable write caching for disk" as this will allow write I/Os
to "end" immediately after the controller gets the data at which time he will DEstage the
data TO disk at his leisure.

The bottom line is getting a faster drive and/or more cache. The cache does work wonders
for boot times as the boot process DOES sort by cyl/head as you suggest to reduce head
seek times.

JL