FileStream.Flush not flushing?

I am writing an applicaiton on the following platform:

Advantech 133Mhz PCM3348.
Windows CE 5.0
Compact Framework.

As part of the application, I am writing information to file (that is,
compact flash) with the expressed intention of protecting it from
unexpected power failures etc.

What I am finding is that FileStream acts very strangely.....

The basic symptom is that sometimes after writing data to the stream
and flushing it, the file size is wrong... eg this works:

fstream.Write(record, 0, record.Length);
fstream.Write(record, 0, record.Length);
fstream.Flush();

(the size of the file is OK - record.Length * 2)

However if I do this:

fstream.Write(record, 0, record.Length);
fstream.Seek(0, SeekOrigin.Current);
fstream.Write(record, 0, record.Length);
fstream.Flush();

or even (!?!?):

fstream.Write(record, 0, record.Length);
fstream.Flush();
fstream.Write(record, 0, record.Length);
fstream.Flush();

the size of the file is only record.Length! WTF?!?
If I close the stream first (fstream.Close()), then it is fine - the
file size if correct.

BTW I measure the size of the file by either using FileInfo, or
rebooting and checking the size of the file. Also these results are
very repeatable - it works the same every time.

This has got me very confused. I don't want to close the stream every
time I write to it. That is expensive!

Any help glady received.

In fact, it flushes only internal buffer but does not touch OS buffer.
To flush OS buffer you must use FlushFileBuffers function. Due to the
fact that FileStream does not use directly CreateFile/WriteFile and
private _handle is incorrect for FlushFileBuffers [1] you have to
implement you own implementation of file stream (that uses CreateFile,
WriteFile, etc.) or just close a stream every time.

[1]
http://msdn.microsoft.com/library/d...FileBuffers.asp

Best regards,
Sergey Bogdanov
http://www.sergeybogdanov.com


Andrew Revell wrote:
> I am writing an applicaiton on the following platform:
>
> Advantech 133Mhz PCM3348.
> Windows CE 5.0
> Compact Framework.
>
> As part of the application, I am writing information to file (that is,
> compact flash) with the expressed intention of protecting it from
> unexpected power failures etc.
>
> What I am finding is that FileStream acts very strangely.....
>
> The basic symptom is that sometimes after writing data to the stream
> and flushing it, the file size is wrong... eg this works:
>
> fstream.Write(record, 0, record.Length);
> fstream.Write(record, 0, record.Length);
> fstream.Flush();
>
> (the size of the file is OK - record.Length * 2)
>
> However if I do this:
>
> fstream.Write(record, 0, record.Length);
> fstream.Seek(0, SeekOrigin.Current);
> fstream.Write(record, 0, record.Length);
> fstream.Flush();
>
> or even (!?!?):
>
> fstream.Write(record, 0, record.Length);
> fstream.Flush();
> fstream.Write(record, 0, record.Length);
> fstream.Flush();
>
> the size of the file is only record.Length! WTF?!?
> If I close the stream first (fstream.Close()), then it is fine - the
> file size if correct.
>
> BTW I measure the size of the file by either using FileInfo, or
> rebooting and checking the size of the file. Also these results are
> very repeatable - it works the same every time.
>
> This has got me very confused. I don't want to close the stream every
> time I write to it. That is expensive!
>
> Any help glady received.

Hmmm. Very interesting.
So when does the OS buffer get updated normally when using a
FileStream? Is it totally up to the OS to decide when to do this?

It seems odd that I would have to bypass .NET to get the behaviour I
want. So this suggests to me that I am using it incorrectly, or I have
a problem understanding the nature of streams. Is it an understood
feature of FileStreams that you cannot guarantee expect the underlying
file to be "correct" until after you close the stream? Is this true of
the full framework / Win32 as well? I always thought Flush wrote the
contents of the buffer to the underlying device - in this case a
file....

Also, as an interesting aside, if I carry out the same experiments on
a file in RAM (just /testfile.dat, not /Mounted Volume/testfile.dat)
the file size is "correct" without closing the stream.


Sergey Bogdanov <sergey.bogdanov@gmail.com> wrote in message news:<Om1n0caOFHA.3828@TK2MSFTNGP10.phx.gbl>...
> In fact, it flushes only internal buffer but does not touch OS buffer.
> To flush OS buffer you must use FlushFileBuffers function. Due to the
> fact that FileStream does not use directly CreateFile/WriteFile and
> private _handle is incorrect for FlushFileBuffers [1] you have to
> implement you own implementation of file stream (that uses CreateFile,
> WriteFile, etc.) or just close a stream every time.
>
> [1]
> http://msdn.microsoft.com/library/d...FileBuffers.asp
>
> Best regards,
> Sergey Bogdanov
> http://www.sergeybogdanov.com
>
>
> Andrew Revell wrote:
> > I am writing an applicaiton on the following platform:
> >
> > Advantech 133Mhz PCM3348.
> > Windows CE 5.0
> > Compact Framework.
> >
> > As part of the application, I am writing information to file (that is,
> > compact flash) with the expressed intention of protecting it from
> > unexpected power failures etc.
> >
> > What I am finding is that FileStream acts very strangely.....
> >
> > The basic symptom is that sometimes after writing data to the stream
> > and flushing it, the file size is wrong... eg this works:
> >
> > fstream.Write(record, 0, record.Length);
> > fstream.Write(record, 0, record.Length);
> > fstream.Flush();
> >
> > (the size of the file is OK - record.Length * 2)
> >
> > However if I do this:
> >
> > fstream.Write(record, 0, record.Length);
> > fstream.Seek(0, SeekOrigin.Current);
> > fstream.Write(record, 0, record.Length);
> > fstream.Flush();
> >
> > or even (!?!?):
> >
> > fstream.Write(record, 0, record.Length);
> > fstream.Flush();
> > fstream.Write(record, 0, record.Length);
> > fstream.Flush();
> >
> > the size of the file is only record.Length! WTF?!?
> > If I close the stream first (fstream.Close()), then it is fine - the
> > file size if correct.
> >
> > BTW I measure the size of the file by either using FileInfo, or
> > rebooting and checking the size of the file. Also these results are
> > very repeatable - it works the same every time.
> >
> > This has got me very confused. I don't want to close the stream every
> > time I write to it. That is expensive!
> >
> > Any help glady received.

Andrew Revell wrote:
> Hmmm. Very interesting.
> So when does the OS buffer get updated normally when using a
> FileStream? Is it totally up to the OS to decide when to do this?
The OS itself decides when an internal buffer should be flushed. It is
done for performance purposes to concatenate small-sized data to a
single piece and when buffer was filled or file was closed OS dumps it
to a file.

> It seems odd that I would have to bypass .NET to get the behaviour I
> want. So this suggests to me that I am using it incorrectly, or I have
> a problem understanding the nature of streams. Is it an understood
> feature of FileStreams that you cannot guarantee expect the underlying
> file to be "correct" until after you close the stream? Is this true of
> the full framework / Win32 as well? I always thought Flush wrote the
> contents of the buffer to the underlying device - in this case a
> file....
Yes, you are right CF.NET must do it for you - I mean call
FlushFileBuffers function when you invoke the Flush() method but as you
can see it doesn't. As I stated before, the only thing that I could
suggest you - Close & Open again FileStream every time when you want to
be sure that all data is in a file.

Best regards,
Sergey Bogdanov
http://www.sergeybogdanov.com