USB Disk speed testing in C#

[sturnfie]

Hi all,

I am attempting to write a tool that would get (via something of a
stopwatch action) the amount of time it takes to copy a file from a
local disk to a connected USB drive on a Windows XP machine.

When I run this tool, I am getting average speeds that are faster than
what is theoretically possible over a full-speed USB connection (the
disk is enumerated at Full speed, not High Speed).

When I simply drag and drop a file, using the computer clock to
crudely time the transfer, the copy process takes much longer than
what my program would indicate (via returned times), but with times
that are much more realistic than what I am seeing. The files I am
transferring are text files of specific sizes that I generate at
runtime.

So my question, is there some form of asynchronous transferring going
on? Does the CopyTo call in my code below kick out before all the
information in the file is transferred? Are there OS optimizations
that are causing havok? This seems to be the case, so how can I test
for when the file is completely copied over?

I chose to use C# for this, but have been reading on this group that
there are weaknesses in the language when it comes to file
manipulation. Any suggestions for better approaches / language
choices would be appreciated.

Here is a bit of what I am currently doing:

private TimeSpan CopyFile(FileInfo localFile, string destinationPath)
{
// Collect the time when we start
DateTime Start = DateTime.Now;

// Copy the file, with overwrite
localFile.CopyTo(destinationPath, true);

// Collect the time when we finish
DateTime stop = DateTime.Now;

// Find the difference
TimeSpan diff = stop.Subtract(start);

// Return difference
return diff;
}


Thank you in advance for any responses

 

[Guest]

Well your "rough timing" code looks OK. Remember that when you drag-drop a
file using Windows Explorer, there is a lot more going on than just a plain
old copy operation. You are getting estimated times being computed, UI
progress being updated, and so on. Gotta compare apples to apples.
Peter

 

[sturnfie]

Well your "rough timing" code looks OK. Remember that when you drag-drop a
file using Windows Explorer, there is a lot more going on than just a plain
old copy operation. You are getting estimated times being computed, UI
progress being updated, and so on. Gotta compare apples to apples.
Peter

I agree. It still burns that the calculation done from this specific
visual "timer" is much more realistic of a value than the result from
the code calculation.

My current attempt is to manually write the files at the byte level.
I can only assume that this would bypass any undocumented / unknown
features that are causing the issues I am seeing.

 

[Andy]

I chose to use C# for this, but have been reading on this group that
there are weaknesses in the language when it comes to file
manipulation. Any suggestions for better approaches / language
choices would be appreciated.

Hi,

The language doesn't matter, as you're using the .Net framework. So
VB.Net would be the same. I'm not sure what posts your referring to
that claim there are 'weaknesses' with file manipulation in C#... it
sounds like a very suspicious statement.

 

[Willy Denoyette [MVP]]

I agree. It still burns that the calculation done from this specific
visual "timer" is much more realistic of a value than the result from
the code calculation.

My current attempt is to manually write the files at the byte level.
I can only assume that this would bypass any undocumented / unknown
features that are causing the issues I am seeing.

What you are measuring is *not* the time it takes to copy a complete file from disk to disk
physically, you are actually measuring the time needed to copy file data to the FS cache,
your timing stops when the file is completed in the cache but before it's actually written
to disk.
Explorer shows you more accurately what's happening because it uses a callback to show
progress, note that even here, it's possible that the FS cache is not completely written to
disk (callbacks are per 64KB on XP) when explorer tells you it has done, note that in case
of a hard disk, it's possible that the HD buffer is not (completely) physically written to
disk before explorer says it has done with the copy.

Willy.

 

[sturnfie]

What you are measuring is *not* the time it takes to copy a complete file from disk to disk
physically, you are actually measuring the time needed to copy file data to the FS cache,
your timing stops when the file is completed in the cache but before it's actually written
to disk.

Thank you Willy, this is extermely helpful. According to filemon
(disk activity moniter), you are absolutely correct.

I suppose that answers my first question, so for a follow-up:

Do you have any suggestions how I can determine the true write time
for copying a file? Would using a Stream object and copying the local
file X bytes at a time to an attached drive give me reasonablly valid
timings? Is there a way to determine when all my information has
exited the FS cache?

 

[Willy Denoyette [MVP]]

Thank you Willy, this is extermely helpful. According to filemon
(disk activity moniter), you are absolutely correct.

I suppose that answers my first question, so for a follow-up:

Do you have any suggestions how I can determine the true write time
for copying a file? Would using a Stream object and copying the local
file X bytes at a time to an attached drive give me reasonablly valid
timings? Is there a way to determine when all my information has
exited the FS cache?

It's not that easy to measure this exactly, even Filemon tells you only when data goes to
the FS, not when the IO manager flushes the cache buffers to disk.
What you can do for instance is use a FileStream (one for reading and one for writing), when
done you should have to PInvoke "FlushFileBuffers" to force the FS to flush it's buffers.
Another option is to use large buffers ( guess > 64KB, but I'm not entirely sure about this)
to prevent File system buffering when writing to disk, note that even in this case I would
suggest you to call FlushFileBuffers when done.

Willy.

PS here the PInvoke declaration. Pass the output FileStream handle as parameter when done
writing.

[DllImport("kernel32", SetLastError = true)]
static extern void FlushFileBuffers(IntPtr handle);

 

[riscy]

Could I see full code on how to transfer file via USB.
Thanks

 

[susiedba]

Andy

C# does not support working with files and folders in a fast manner

Windows Script Host works well enough so who gives a **** about some
new-fangled language?

 

[RobinS]

Ignore the susiedba post; he's trolling.
Robin S.
-----------------------------------------