copying files

[Analizer1]

Ive Written a static FileCopy
from all your experiences , what would be a good Buffer size Byte[]
array..for best performance..

I know there is a built in file.copy ..I need to put progress bar up,
because some files are pretty huge

Tks

 

[Jon Skeet [C# MVP]]

Ive Written a static FileCopy
from all your experiences , what would be a good Buffer size Byte[]
array..for best performance..

I know there is a built in file.copy ..I need to put progress bar up,
because some files are pretty huge

I usually go between 8K and 64K. Going much above that you won't see
much benefit and you'll end up with objects on the Large Object Heap.

Have you written a benchmark for this? Bear in mind that things like
the file cache make it hard to test accurately.

 

[Jeroen Mostert]

Ive Written a static FileCopy
from all your experiences , what would be a good Buffer size Byte[]
array..for best performance..

None. Since you're not processing the data, a buffer would just be a waste.
Letting the OS handle the copying in its entirety is best.

I know there is a built in file.copy ..I need to put progress bar up,
because some files are pretty huge

Aha. If that's the problem, you can use

Microsoft.VisualBasic.FileIO.FileSystem.CopyFile(source, destination,
UIOption.AllDialogs);

Yes, I know that's a mouthful and the VisualBasic part is confusing, but
don't be put off by that, it has what you need. It uses the standard file
copy functionality of the shell. Don't forget to add a reference to the
Microsoft.VisualBasic assembly.

 

[Analizer1]

id Rather use, FileStream, More options i can code in the lowest level

thanks though

Ive Written a static FileCopy
from all your experiences , what would be a good Buffer size Byte[]
array..for best performance..

None. Since you're not processing the data, a buffer would just be a
waste. Letting the OS handle the copying in its entirety is best.

I know there is a built in file.copy ..I need to put progress bar up,
because some files are pretty huge

Aha. If that's the problem, you can use

Microsoft.VisualBasic.FileIO.FileSystem.CopyFile(source, destination,
UIOption.AllDialogs);

Yes, I know that's a mouthful and the VisualBasic part is confusing, but
don't be put off by that, it has what you need. It uses the standard file
copy functionality of the shell. Don't forget to add a reference to the
Microsoft.VisualBasic assembly.

 

[Analizer1]

im gonna do some testing in a bit using multi threads
and 2 different computers.. and file sizes are about
500mg

thanks alot

Ive Written a static FileCopy
from all your experiences , what would be a good Buffer size Byte[]
array..for best performance..

I know there is a built in file.copy ..I need to put progress bar up,
because some files are pretty huge

I usually go between 8K and 64K. Going much above that you won't see
much benefit and you'll end up with objects on the Large Object Heap.

Have you written a benchmark for this? Bear in mind that things like
the file cache make it hard to test accurately.

 

[Jeroen Mostert]

id Rather use, FileStream, More options i can code in the lowest level

Well, if you want to reinvent the wheel, I can't stop you.

That said, if you want "the lowest level", you can P/Invoke to CopyFileEx
and use your own progress routines. It doesn't get much lower than that.
Here's a good article on that:
http://msdn.microsoft.com/msdnmag/issues/05/02/NETMatters/

Do not for a moment think that you will get the best performance or
reliability by using the managed FileStream, though. Reading and writing
chunks of data in managed code is much slower than letting the OS do the
buffering and transfer of data directly, because you have to transition from
managed code to unmanaged code to kernel mode and back again for every
chunk of copied data. That said, depending on your needs, it may be "fast
enough", and it's certainly easier to write.

 

[Jon Skeet [C# MVP]]

im gonna do some testing in a bit using multi threads
and 2 different computers.. and file sizes are about
500mg

Rather than explicitly using multiple threads, consider using
asynchronous IO.

 

[Arne Vajhøj]

Ive Written a static FileCopy
from all your experiences , what would be a good Buffer size Byte[]
array..for best performance..

I would go for 100 KB.

Big, but not bigger than the data will still be in L2 cache (my PC
only have 1 MB L2 cache). The benefits of bigger buffer size
is decreasing anyway.

I once did a little test and got the following results:

Buffer: 10 Time: 4406
Buffer: 100 Time: 2093
Buffer: 1000 Time: 1625
Buffer: 10000 Time: 718
Buffer: 100000 Time: 453
Buffer: 1000000 Time: 750
Buffer: 10000000 Time: 765

Arne

 

[Analizer1]

ok...i'll test with async also....i'll post my findings in a lil while

thanks

 

[Analizer1]

thanks Arne

Ive Written a static FileCopy
from all your experiences , what would be a good Buffer size Byte[]
array..for best performance..

I would go for 100 KB.

Big, but not bigger than the data will still be in L2 cache (my PC
only have 1 MB L2 cache). The benefits of bigger buffer size
is decreasing anyway.

I once did a little test and got the following results:

Buffer: 10 Time: 4406
Buffer: 100 Time: 2093
Buffer: 1000 Time: 1625
Buffer: 10000 Time: 718
Buffer: 100000 Time: 453
Buffer: 1000000 Time: 750
Buffer: 10000000 Time: 765

Arne