BinaryReader, PeekChar, Conversion buffer overflow

[IGW]

I am getting a "conversion buffer overflow" in PeekChar
method of a BinaryReader.

(I am using C++, VS .NET 2003).

Always seems to be in the same place in any given file but
the location of the error varies according to the file.

I know that I can do the read in a try catch and/or other
things to solve the issue - that is not the point.
PeekChar should work.

Any ideas,
Ian

Code in question looks like (button handler and the word
wrap may not be ideal):

private: System::Void btnGo_Click(System::Object *
sender, System::EventArgs * e)
{
FileStream * fIn;
FileStream * fOut;
BinaryReader * In;
BinaryWriter * Out;
unsigned char CurrentByte;

if(File::Exists(tbInFile->Text))
{
fIn = new FileStream(tbInFile->Text,
FileMode::Open);
fOut = new FileStream(tbFileOut->Text,
FileMode::Create);
In = new BinaryReader(fIn);
Out = new BinaryWriter(fOut);

while (In->PeekChar() != -1) // bombs here
{
CurrentByte = In->ReadByte();
// do stuff here
// ...
Out->Write(CurrentByte);

// while debugging PeekChar issue
// flush byte by byte
Out->Flush();
}

Out->Flush();
Out->Close();
In->Close();
fIn->Close();
fOut->Close();
}
}
};

 

[Jon Skeet]

I am getting a "conversion buffer overflow" in PeekChar
method of a BinaryReader.

(I am using C++, VS .NET 2003).

Always seems to be in the same place in any given file but
the location of the error varies according to the file.

I know that I can do the read in a try catch and/or other
things to solve the issue - that is not the point.
PeekChar should work.

My guess is that your file contains something which isn't an
appropriate character in UTF-8. Not a terribly nice exception though.

I'm confused as to exactly why you're reading in this particular way
though. Is the aim just to copy the file? If so, there are much better
ways of doing it. (You should also consider doing it in a different
thread so as to avoid the GUI locking up.)

 

[IGW]

My guess is that your file contains something which isn't an
appropriate character in UTF-8. Not a terribly nice

exception though.

I am an embedded sort of person. If I read a byte I
expect to be able to read anything. PeekChar doco makes
no mention of any problem in representing characters or
character sets.

PeekChar returns an int which is quite capable of
representing *any* 8 bit value, while still being able to
return -1 as a int.

The problem is not simply dependent on the character
either, the same character eralier in the file does not
cause any problem. But for any one file the exception
always seems to occur in the same file position. Different
characters and positions though in different files. (In
the two files I have tested so far it is around file
offset 5600 to 5800 (decimal).)

OK, here is another question, what encoding should I use
to be able to read *any* 8-bit character? This is a
binary reader after all.

I'm confused as to exactly why you're reading in this particular way
though. Is the aim just to copy the file? If so, there are much better
ways of doing it. (You should also consider doing it in a different
thread so as to avoid the GUI locking up.)

Very quick and dirty to insert bit errors into files at a
user specified probabalistic bit error rate (eg 1 in
1000). Used for testing a few ideas on a communication
system.

Not production code. I agree that I could do this many
different ways - including not in .NET but this was used a
as a little safe learning exercise for a .NET newbie. I
have no issue with getting the program doing what is
needed but maybe someone out there is interested in
investigating a possible bug. Treat this as a possible
bug report rather than a request for help if you wish.

Thanks,
Ian

 

[IGW]

-----Original Message-----


But you weren't reading a byte. You were peeking a *character*. A
character isn't the same as a byte.

Got no issue understanding this (byte v char).

The BinaryReader does not provide a means to peek a byte,
and ReadByte does not return any indication of end-of-
stream - it generates an EndOfStreamException, just a
(very little) more pain to deal with (but what I did in
the end, just read bytes until I got the exception).

(As described by a colleague, using exceptions for
general, expected, flow of control situations, is a little
like p*ssing in your pants to see if your fly is open.
Especially when you take into account the overhead
exception processing imposes. I apologise for the
crudeness of the analogy.)

The use of PeekChar in this fashion came from some sample
I saw - possibly on codeguru or codeproject??

As for reading big buffers in one go and mucking about
with them, yes this is what I do in critical code we use
on protocol engines - this is not such a case, just
playing with .NET on a simple hack program.

Bottom line is: PeekChar should not cause such an
exception even though my implementation is sub-optimal.

Pretty much done here I think, thanks for the responses,
Ian

 

[Jon Skeet]

The BinaryReader does not provide a means to peek a byte,
and ReadByte does not return any indication of end-of-
stream - it generates an EndOfStreamException, just a
(very little) more pain to deal with (but what I did in
the end, just read bytes until I got the exception).

Ah yes - I'd forgotten that you were using BinaryReader rather than
just a straight stream.

(As described by a colleague, using exceptions for
general, expected, flow of control situations, is a little
like p*ssing in your pants to see if your fly is open.
Especially when you take into account the overhead
exception processing imposes. I apologise for the
crudeness of the analogy.)

The use of PeekChar in this fashion came from some sample
I saw - possibly on codeguru or codeproject??

If it's using PeekChar on a binary file, I'd say that's flawed. Yes,
you *could* do it having set the encoding to ISO-8859-1 or some other
hack, but it's still basically treating binary data as text, which I
don't think is a good idea.

As for reading big buffers in one go and mucking about
with them, yes this is what I do in critical code we use
on protocol engines - this is not such a case, just
playing with .NET on a simple hack program.

In that case just use a plain Stream instead of BinaryReader -
Stream.ReadByte will return -1 at the end of the stream. Mind you, it's
not sufficiently hard to do it on a block basis that I'd think it worth
avoiding anyway.

Bottom line is: PeekChar should not cause such an
exception even though my implementation is sub-optimal.

I think it's reasonable for PeekChar to throw an exception if it comes
across a byte sequence which doesn't represent a valid character.
Probably not the exception it sounds like it *did* throw though.

 

[epsilonsdad]

Great Thread

Just want to say thanks to the group.

This thread helped me resolve the same problem with the BinaryReader and PeekChar.