Perplexed on logging both text and binary information to a file

[Andrew Falanga]

Hi,

On the heals of my previous post I'm now asking for the proper way of
writing text and binary data to a log file. I have a class, in the
singleton pattern, that will be used for logging program operations to
a file. I'm having trouble with using the two.

I eventually ended up at a point of using a FileStream object as a
"base" and then, in my class, also having a BinaryWriter and a
StreamWriter object both taking the FileStream object as an argument
for construction. I ended up with some very odd results when trying
to use these two objects in tandem.

I then noticed that the BinaryWriter object had member functions that
took string objects and such and decided to use only a BinaryWriter
object. This has proven unproductive too. The string objects I pass
are being written to the log file as text but there is extra
characters in the mix. What am I missing? Is this something with the
encoding of the file (UTF8 and so forth)?

I expect that I will be primarily logging text to this log file, but I
can see that the need can arise, as I write this application, that
writing binary data will be necessary. I need to write it into the
same file. Any help is most appreciated.

Thanks,
Andy

 

[Jeff Johnson]

On the heals of my previous post I'm now asking for the proper way of
writing text and binary data to a log file. I have a class, in the
singleton pattern, that will be used for logging program operations to
a file. I'm having trouble with using the two.

I eventually ended up at a point of using a FileStream object as a
"base" and then, in my class, also having a BinaryWriter and a
StreamWriter object both taking the FileStream object as an argument
for construction. I ended up with some very odd results when trying
to use these two objects in tandem.

I then noticed that the BinaryWriter object had member functions that
took string objects and such and decided to use only a BinaryWriter
object. This has proven unproductive too. The string objects I pass
are being written to the log file as text but there is extra
characters in the mix. What am I missing? Is this something with the
encoding of the file (UTF8 and so forth)?

I expect that I will be primarily logging text to this log file, but I
can see that the need can arise, as I write this application, that
writing binary data will be necessary. I need to write it into the
same file. Any help is most appreciated.

Who is this log file for? Logs are traditionally text-based so that they can
be read by humans. If you need to log some non-text data, you often do so
with a "hex dump."

If you DO want to log binary data but also have human-readable text in
there, I recommend that you use an Encoding class. Encoding.Default is the
best choice in my opinion because it will return the default ANSI encoding
for your system locale. Then you can run your string through the GetBytes()
method and get back an array of bytes which, when written to your log,
should show up without any extra "fluff" when viewed in a text editor.

On the other hand, if you want to write everything as plain text, let me
know, because I have a routine which creates hex dumps and I'll post it for
you.

 

[Phil Wilson]

Something that sometimes works if you have binary data that is wrapped in an
object is to write your own ToString() method to turn the content into a
human readable string.

 

[Arne Vajhøj]

On the heals of my previous post I'm now asking for the proper way of
writing text and binary data to a log file. I have a class, in the
singleton pattern, that will be used for logging program operations to
a file. I'm having trouble with using the two.

I eventually ended up at a point of using a FileStream object as a
"base" and then, in my class, also having a BinaryWriter and a
StreamWriter object both taking the FileStream object as an argument
for construction. I ended up with some very odd results when trying
to use these two objects in tandem.

I then noticed that the BinaryWriter object had member functions that
took string objects and such and decided to use only a BinaryWriter
object. This has proven unproductive too. The string objects I pass
are being written to the log file as text but there is extra
characters in the mix. What am I missing? Is this something with the
encoding of the file (UTF8 and so forth)?

I expect that I will be primarily logging text to this log file, but I
can see that the need can arise, as I write this application, that
writing binary data will be necessary. I need to write it into the
same file. Any help is most appreciated.

Always log text. If you have binary data you need to log then
convert them to hex string first.

Arne

 

[Andrew Falanga]

Who is this log file for? Logs are traditionally text-based so that they can
be read by humans. If you need to log some non-text data, you often do so
with a "hex dump."

For people to read. Though I expect to log nearly always text, for
the purpose this program must fill, I might have cases where I'll have
to log some of the data that I'm reading from the input sources
(serial ports and the like). This data won't always contain text and
I can see the need to log non-text data for analysis of issues. At
this point, I'm more concerned with having the functionality in place
so as to be able to do it when I get into coding the "actual" code of
the program.

If you DO want to log binary data but also have human-readable text in
there, I recommend that you use an Encoding class. Encoding.Default is the
best choice in my opinion because it will return the default ANSI encoding
for your system locale. Then you can run your string through the GetBytes()
method and get back an array of bytes which, when written to your log,
should show up without any extra "fluff" when viewed in a text editor.

One of my attempts, before posting, was doing something like this and
then passing that byte array to a StreamWriter object (I think). I'm
not sure of the web page I was on with that code example and I'm very
much a novice at all this (my background is entirely UNIX and C++).
Can you provide an example of how this would be done?

On the other hand, if you want to write everything as plain text, let me
know, because I have a routine which creates hex dumps and I'll post it for
you.

If the byte arrays, encoding objects and GetBytes() routines aren't
adequate or the right way to do this, that may be helpful. Thanks for
the offer.

Andy

 

[Tim Jarvis]

Always log text. If you have binary data you need to log then
convert them to hex string first.

Arne

I totally agree with Arne on this, you might also try Base64 for your
binary data.

 

[Arne Vajhøj]

I totally agree with Arne on this, you might also try Base64 for your
binary data.

If the log is to be read by a program, then Base64 will save 1/3
compared to hex.

But if it is to be read by humans, then hex is 1000 times easier
to read than Base64.

Arne

 

[Jeff Johnson]

One of my attempts, before posting, was doing something like this and
then passing that byte array to a StreamWriter object (I think). I'm
not sure of the web page I was on with that code example and I'm very
much a novice at all this (my background is entirely UNIX and C++).
Can you provide an example of how this would be done?

How about this: can you post the code you're currently using to write
strings? Then we can dissect it.

 

[Andrew Falanga]

How about this: can you post the code you're currently using to write
strings? Then we can dissect it.

Sorry I did not reply to this sooner. The place I work has some
rather restrictive rules regarding source code. So, I can't post.
However, I've managed to solve the problem. I've been able to
adequately accomplish what I needed by using a FileStream object and
using the following (actually taken from MSDN) to accomplish the
majority of what I need:

// assume this is part of a properly defined class
// with a FileStream member object

public void Log(String s) {
// using the classes FileStream member fs
byte[] tmp = new UTF8Encoding(true).GetBytes(s);
fs.Write(tmp, 0, fs.Length);
fs.Flush();
}

// and then, for writing the binary
public void WriteBytes(byte[] ba) {
fs.Write(ba, 0, fs.Length);
fs.Flush();
}

// and then, for writing the bytes of a byte array as text
public void WriteBytesAsText(byte[] ba) {
String tempStr = "";
for(int i = 0; i < ba.Length; i++)
tempStr += ba.ToString();

byte[] tempBA = new UTF8Encoding(true).GetBytes(tempStr);
fs.Write(tempBA, 0, tempBA.Length);
fs.Flush();
}


Basically, I use a similar process as above to convert the byte array
into hex characters using System.Convert.ToString(byte val, int base)
when I'd like to dump the contents as hex text.

Thanks for the help here and in the other thread I posted. It was
these threads that helped me solve this problem in part. I would
welcome any suggestions on how to better do this.

Andy