Stream question. Why won't this work without close?

[Guest]

Greetings all...

I was playing around with compressing streams and came across a behavior
that I do not understand. I create a stream (input) from the contents of a
textbox. That stream is compressed into another stream (output). I then
copy the stream (output) to another stream (input2). The compressed stream
(input2) is then decompressed into a final stream (output2).

My question is this. I create input2 from output.GetBuffer() method. If
I close the compress DeflateStream prior to the copy, everything works. If I
don't close it, the decompression won't work. If I comment out the Close(),
I can see no difference using watch. Can someone help me understand why I
have to use the close?

Here is the code...


private void button1_Click(object sender, EventArgs e)
{
MemoryStream input = new
MemoryStream(UnicodeEncoding.Unicode.GetBytes(textBox1.Text));
MemoryStream output = new MemoryStream();

Stream compress = new DeflateStream(output,
CompressionMode.Compress);
int theByte = input.ReadByte();
while (theByte != -1)
{
compress.WriteByte((byte)theByte);
theByte = input.ReadByte();
}

//Why does this have to be here?
compress.Close();

MemoryStream output2 = new MemoryStream();
MemoryStream input2 = new MemoryStream(output.GetBuffer());


Stream decompress = new DeflateStream(input2,
CompressionMode.Decompress);
theByte = decompress.ReadByte();
while (theByte != -1)
{
output2.WriteByte((byte)theByte);
theByte = decompress.ReadByte();
}

textBox2.Text =
UnicodeEncoding.Unicode.GetString(output2.GetBuffer());


}

Thanks in advance for any help.

 

[Marc Gravell]

The compression (as with many stream operations) is buffered - i.e. it
writes chunks down; most likely, the compression stream still has a
leftover bit of unwritten data in its memory.

You could try adding .Flush() in place of .Close() - this is meant to
force the stream to commit its buffers to the underlying stream;
however, a while ago (with a similar issue) I did manage to demonstrate
that at least one common compression stream did not fully respect
..Flush() [presumably due to some optimisation relating to compression -
i.e. it demands bigger chunks]. In this instance, I had to .Close() the
outer stream before it would commit its last bytes, using the
overloaded compression ctor that tells the outer stream not to .Close()
the inner stream when the outer stream is .Close()d.

Marc

 

[Marc Gravell]

Other points; you could actually be doing too much:
MemoryStream.GetBuffer() returns the current raw memory buffer, which
is usually oversized - i.e. if your stream's length is 1050 bytes,
GetBuffer() may return a 2048 byte array. You need to either limit
yourself to the first .Length bytes, else call .ToArray(), which
returns a right-sized buffer (i.e. 1050 bytes).

Also: ReadByte() is doing this the intensive way; normally I would
expect to see blocked transfers using a buffer of (for instance) 512
bytes, 1024 btes, something like that; otherwise you make a *lot* of
calls. It is for this reason that stream operations typically perform
buffering internally, so that they can retain efficiency. It'll work
(if you fix the top point), but it could turn out to be a performance
pinch-point. As an example, the following is a typical "copy stream A
to stream B" method (using a user-defined buffer-size):

public static long Copy(System.IO.Stream source, System.IO.Stream
destination, int bufferSize) {
long totalBytesRead = 0;
byte[] buffer = new byte[bufferSize];
int bytesRead;
while ((bytesRead = source.Read(buffer, 0, bufferSize)) >
0) {
destination.Write(buffer, 0, bytesRead);
totalBytesRead += bytesRead;
}
destination.Flush();
return totalBytesRead;
}

Hope this helps,

Marc

 

[Marc Gravell]

Final reply (unless there is a question); if (as I suggested) you don't
close the inner stream, you can actually re-use the inner stream, by
[ab]using the fact that memory-streams are seekable. Generally streams
should not be considered seekable, but this one is...

so then you could have (pseudo-code):

create base (memory) stream
create compression-stream on top of base stream (set to not close the base
stream)
write data to the compression stream
close the compression stream (flushes everything to the base stream)
rewind the base stream (set position = 0)
create decompression-stream on top of base stream (set to not close the base
stream)
read data from the decompression stream
close the decompression stream
close the base stream

The advantage here is that we have re-used the data memory stream between
uses, without having to go near .GetBuffer() or .ToArray().

Marc

 

[Guest]

Marc,

Thanks for all of the info. Some of the things I was doing in the code
(like transferring one byte at a time) was so I could examine more carefully
how things were working. Normally I would transfer chunk data.

Most of what you said was along the lines of what I was thinking. I just
don't have enough experience in this area and wanted to confirm it. Most
books treat streams too much like black boxes and don't get into how they
actually work.

Thanks again,
Scott