memory mapped file

[sunil]

Hi,

I am trying to write large amount of data to 10 GB to csv file. If i
use memory-mapped file would it help in increasing the performance? I
am planning to dump 256 MB to file.

Regards,
sunhcl

 

[Andreas Johansson]

How are you planning to write to the file? Most IO is buffered so adding an
extra "buffer" by writing to a MemoryStream would most likely not increase
performance.

 

[sunil]

http://www.linuxtopia.org/online_books/programming_books/thinking_in_java/TIJ314_029.htm

The above link suggests that it will improve the performance with
memory-mapped file.

Does any-body can confirm would i get the similar kind of improvement
in windows programming too.

 

[Stephen Martin]

The relative performance of using a FileStream as opposed to memory-mapped
files basically comes down to usage patterns and hardware capabilities, not
so much the size of the file or the data you are accessing.

In the majority of cases file access is sequential or localized and a stream
based approach with the appropriate buffer size is definitely the better
method and should out perform memory-mapped files in virtually all cases. On
the other hand when your file access is random (if you find yourself using
Seek or Position frequently to move significant distances) then
memory-mapped files will usually out-perform stream based implementations,
sometimes by large margins.

Memory-mapped files are not directly supported by the .Net framework but are
relatively easy to implement using unmanaged code.

Though, from the sounds of your post you will be doing sequential type
access so the stream based approach is almost certainly superior for you.