PC Review


Reply
Thread Tools Rate Thread

Built in compression on .net looses data, maybe?

 
 
TerryStone
Guest
Posts: n/a
 
      29th Aug 2006
Thanks to anyone who reads this.

Below is some C# that compresses an array of bytes, and then
decompresses, and compares the original data with the new.

Firstly, the length of the decompressed data is shorter than the
original. So some loss of data has occured. But the content up until
the early truncation matches. So am I flushing correctly? This error
only occurs for particular combinations of bytes in the original
buffer.

Secondly, when I read the decompressed data from the zip-stream, the
first read returns zero bytes. After that I perform a second read and
the data can be read. Why is that?

using System;
using System.Collections.Generic;
using System.Diagnostics;
using System.IO;
using System.IO.Compression;
using System.Text;

namespace lab02
{

class Program
{

static void Main(string[] args)
{

// declaration of local variables
int i1 = 0;

// create a buffer of data for compressing
byte[] bufferData = new byte[10];
for (i1 = 0; i1 < 10; i1++)
bufferData[i1] = Convert.ToByte(i1);

// PART 1 - Compression
byte[] bufferCompressed = null;
{

// compress buffer into a memory stream (ms)
MemoryStream msCompressed = new MemoryStream();
DeflateStream zipStream = new DeflateStream(msCompressed,
CompressionMode.Compress);
zipStream.Write(bufferData, 0, bufferData.Length);
msCompressed.Flush();

// get the compressed memory stream into a buffer
bufferCompressed = new byte[msCompressed.Length];
msCompressed.Position = 0;
msCompressed.Read(bufferCompressed, 0, bufferCompressed.Length);

// close zip stream
zipStream.Close();

}

// PART 2 - Decompression
byte[] bufferDecompressed = null;
{

// put the compressed data (bufferCompressed) into a memory stream
(msCompressed)
MemoryStream msCompressed = new MemoryStream();
msCompressed.Write(bufferCompressed, 0, bufferCompressed.Length);
msCompressed.Position = 0;

// decompress buffer
DeflateStream zipStream = new DeflateStream(msCompressed,
CompressionMode.Decompress);
msCompressed.Flush();

// read the de-compressed data into a buffer
MemoryStream msDecompressed = new MemoryStream();
int iBytesRead = 0;
byte[] bufferSub = new Byte[1024];
do
{

// read next bytes (problem with the first read, always read zero
first)
iBytesRead = zipStream.Read(bufferSub, 0, bufferSub.Length);
if ((msDecompressed.Length == 0) && (iBytesRead == 0))
iBytesRead = zipStream.Read(bufferSub, 0, bufferSub.Length);

// if some data was read...
if (iBytesRead > 0)
{

// add to stream
msDecompressed.Write(bufferSub, 0, iBytesRead);

}

} while (iBytesRead == bufferSub.Length);

// close zip stream
zipStream.Close();

// load buffer with unzipped data
bufferDecompressed = new byte[msDecompressed.Length];
msDecompressed.Position = 0;
msDecompressed.Read(bufferDecompressed, 0,
bufferDecompressed.Length);

}

// PART 3 - Comparision of what was and now is, or is it?
if(bufferData.Length!=bufferDecompressed.Length)
Trace.TraceInformation("Length mismatch!!!");
else
{

// compare contents
bool bMatch = true;
for (i1 = 0; i1 < bufferData.Length; i1++)
{

// if bytes do not match...
if (bufferData[i1] != bufferDecompressed[i1])
{

// update flag
bMatch = false;

// break out of loop
break;

}

}
if(!bMatch)
Trace.TraceInformation("Content does not match!!!");

}

}

}

}

 
Reply With Quote
 
 
 
 
Marc Gravell
Guest
Posts: n/a
 
      29th Aug 2006
First - you are flushing the wrong stream; it is zipStream that needs
flushing. However, I have seen the compression classes refuse to Flush
completely until Close is called; presumably this is an optimisation to keep
a few bytes for use in the compression algorithm. So My compression code
would be (note overload to ctor to leave the stream open):

using (MemoryStream msCompressed = new MemoryStream()) {
using (DeflateStream zipStream = new
DeflateStream(msCompressed, CompressionMode.Compress, true)) {
zipStream.Write(bufferData, 0, bufferData.Length);
zipStream.Close();
}
bufferCompressed = msCompressed.ToArray();
}

I don't know why your code reports zero (I didn't even run it to find out,
I'm afraid) - however, I would do as follows; note the trick is on the while
condition (which captures the count and tests it in one go).

using (MemoryStream msDecompressed = new MemoryStream()) {
using (MemoryStream msCompressed = new
MemoryStream(bufferCompressed))
using (DeflateStream zipStream = new
DeflateStream(msCompressed, CompressionMode.Decompress)) {
int bytesRead;
const int BUFFER_SIZE = 1024;
byte[] buffer = new byte[BUFFER_SIZE];
while ((bytesRead = zipStream.Read(buffer, 0,
BUFFER_SIZE)) > 0) {
msDecompressed.Write(buffer, 0, bytesRead);
}
}
bufferDecompressed = msDecompressed.ToArray();
}

Third - in this scenario, since I used the ctor to not close the stream
(first block of code) I could actually re-use my MemoryStream between the
two blocks simply by rewinding (.Position = 0); but the above works fine,
and illustrates the point. Also note that short or fairly random blocks of
data can get longer during compression. Ironic, but life, especially with
single-pass compression. Double-pass compression can use a "don't bother"
flag.

Marc


 
Reply With Quote
 
TerryStone
Guest
Posts: n/a
 
      29th Aug 2006
Thanks for the help Marc. Especially the

Stream.ToArray();

function, which make the code a lot more compact and readable.

I have discovered from another group that the zip stream must be
flushed AND closed before the compressed data can be read.

zipStream.Flush();
zipStream.Close();

Solved!

Terry.

 
Reply With Quote
 
 
 
Reply

Thread Tools
Rate This Thread
Rate This Thread:

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
BCM B2TR looses data! =?Utf-8?B?TGlvbk5ldA==?= Microsoft Outlook BCM 0 25th Sep 2006 05:53 PM
XP built-in compression feature jackmmr Windows XP General 0 11th Sep 2006 09:42 PM
Windows XP's built in compression utility Bryan Dickerson Windows XP General 4 25th May 2005 06:27 PM
IIS built-in compression Just D. Microsoft ASP .NET 10 15th Mar 2005 09:24 PM
System Looses Data Thulani Windows XP Logo 0 22nd Apr 2004 01:39 PM


Features
 

Advertising
 

Newsgroups
 


All times are GMT +1. The time now is 08:54 AM.