Zip file using a stream

[Guest]

I am trying to make a minor modification to the code below and need some
assistance. Currently this code is using the java.util, java.util.zip, and
java.io assemblies from the vjslib.dll assembly. This code works fine by
taking file(s) from
disk and creating a zip file to disk. I need to modify this code to read a
file from a byte array and then output the zip file to a byte array (all done
in memory instead of disk). The reason for the byte array is that all the
files are being stored in the database in a BLOB field. Any assistance is
appreciated!

private static void Zip(string zipFileName, string[] sourceFile)
{
// TODO: this contains the file retrieved from the database.
Need to load this into the FileOutputStream.
byte[] ba = GetBlob(13); // getblob is a method that queries the DB

// this prepares a zip file for output to disk
FileOutputStream filOpStrm = new FileOutputStream(zipFileName);
ZipOutputStream zipOpStrm = new ZipOutputStream(filOpStrm);

FileInputStream filIpStrm = null;
foreach(string strFilName in sourceFile)
{
// load the file contents into a system.io.filestream
filIpStrm = new FileInputStream(strFilName);
ZipEntry ze = new ZipEntry(Path.GetFileName(strFilName));
zipOpStrm.putNextEntry(ze);
sbyte[] buffer = new sbyte[1024];
int len = 0;
while ((len = filIpStrm.read(buffer)) >= 0)
{
// this writes zip file to disk
//TODO: need to write this stream to a byte array instead of
disk, which I'll insert into the BLOB field.
zipOpStrm.write(buffer, 0, len);

}
}
zipOpStrm.closeEntry();
filIpStrm.close();
zipOpStrm.close();
filOpStrm.close();
}

 

[Jon Skeet [C# MVP]]

I am trying to make a minor modification to the code below and need some
assistance. Currently this code is using the java.util, java.util.zip, and
java.io assemblies from the vjslib.dll assembly. This code works fine by
taking file(s) from
disk and creating a zip file to disk. I need to modify this code to read a
file from a byte array and then output the zip file to a byte array (all done
in memory instead of disk). The reason for the byte array is that all the
files are being stored in the database in a BLOB field. Any assistance is
appreciated!

Use a MemoryOutputStream instead of a FileOutputStream. You can do this
without using the Java classes at all using #ZipLib (google SharpZipLib
for details) and MemoryStream.

 

[Guest]

Jon,

Thank You for your advice. I've switched to the ZipLib and have a few issues.
#1 The last file is not being added to the ZipOutputStream entries
#2 When I test the zip file by writing the stream to disk, it is 0 length,
but valid.

Any assistance you can provide is appreciated

using System;
using System.Collections.Generic;
using System.Text;
using System.IO;
using System.Data.SqlClient;
using System.Configuration;
using System.Data;
using ICSharpCode.SharpZipLib;
using ICSharpCode.SharpZipLib.Zip;

namespace testzipper
{
class Program
{
static void Main(string[] args)
{

string[] files = new string[] { "c:\\temp\\test1.txt",
"c:\\temp\\test2.txt", "c:\\temp\\test5.pdf", "c:\\temp\\test6.doc" };
ZipToMemory("test.zip", files);
}

// this method outputs a zip file to memory
private static void ZipToMemory(string zipFileName, string[]
filenames)
{
try
{

System.IO.MemoryStream ms = new MemoryStream();
using (ZipOutputStream s = new ZipOutputStream(ms))
{

s.SetLevel(9); // 0 - store only to 9 - means best
compression
byte[] buffer = new byte[4096];
foreach (string file in filenames)
{
ZipEntry entry = new ZipEntry(file);
entry.DateTime = DateTime.Now;
s.PutNextEntry(entry);
// TODO: last file is not added as entry
byte[] ba = GetBlob(13);
s.Write(ba, 0, ba.Length);
}

// TODO: write stream to BLOB
//...But before I try that test that the zip is valid by
writting the ZipOutputStream
// to disk
using (FileStream fs = File.OpenRead(zipFileName))
{
int sourceBytes;
do
{
sourceBytes = fs.Read(buffer, 0, buffer.Length);
s.Write(buffer, 0, sourceBytes);
} while (sourceBytes > 0);
}

s.Finish();
s.Close();
}

}
catch (Exception ex)
{
Console.WriteLine("Exception during processing {0}", ex);

}
}

private static byte[] GetBlob(int id)
{
byte[] data = null;
string fileName = string.Empty;
string cn = "xxxxxxxx";
using (SqlConnection mySqlConnection = new SqlConnection(cn))
{
SqlCommand myCommand = new SqlCommand("BN_GetData",
mySqlConnection);
myCommand.CommandType = CommandType.StoredProcedure;

SqlParameter prmId = new SqlParameter("@ID", SqlDbType.Int);
prmId.Value = id;
myCommand.Parameters.Add(prmId);

mySqlConnection.Open();
using (SqlDataReader myReader = myCommand.ExecuteReader())
{
if (myReader.Read())
{
//
data =
(byte[])myReader[myReader.GetOrdinal("PDFData")];
fileName = (string)myReader["FileName"];
}
myReader.Close();
}
mySqlConnection.Close();
}
return data;
}
}
}

 

[Jon Skeet [C# MVP]]

Thank You for your advice. I've switched to the ZipLib and have a few issues.
#1 The last file is not being added to the ZipOutputStream entries
#2 When I test the zip file by writing the stream to disk, it is 0 length,
but valid.

Any assistance you can provide is appreciated

Not sure - but I'd strongly suggest writing a smaller test app that
doesn't need to go to the database. It's much easier to isolate the
problem when it's just local. (It'll also be easier for me to
investigate.)

 

[Peter Duniho]

Jon,

Thank You for your advice. I've switched to the ZipLib and have a few issues.
#1 The last file is not being added to the ZipOutputStream entries
#2 When I test the zip file by writing the stream to disk, it is 0 length,
but valid.

In addition to what Jon wrote, I'm wondering if the ZipOutputStream
class has some internal buffering and provides a Flush() method or
something like that.

I've never used it myself, but the symptoms you describe sound
consistent with failing to flush some intermediate buffer.

Thought, in the case of the #2 problem, it's not clear to me what you
mean when you write "it is 0 length, but valid". How can a 0 length
file be a valid ZIP file?

Pete

 

[Peter Duniho]

And one more thing...

[...]
byte[] buffer = new byte[4096];
foreach (string file in filenames)
{
ZipEntry entry = new ZipEntry(file);
entry.DateTime = DateTime.Now;
s.PutNextEntry(entry);
// TODO: last file is not added as entry
byte[] ba = GetBlob(13);
s.Write(ba, 0, ba.Length);
}

This part seems okay, at least without knowing anything specific about
the class, and keeping in mind the possibility that you need to call
some sort of flush method.

But this part confuses me:

// TODO: write stream to BLOB
//...But before I try that test that the zip is valid by
writting the ZipOutputStream
// to disk
using (FileStream fs = File.OpenRead(zipFileName))
{
int sourceBytes;
do
{
sourceBytes = fs.Read(buffer, 0, buffer.Length);
s.Write(buffer, 0, sourceBytes);
} while (sourceBytes > 0);
}

What exactly are you trying to do here? What is the contents of the
"test.zip" file? What is the point of appending those contents to your
ZipOutputStream? And why does it make sense to append those contents to
the last ZipEntry in the ZipOutputStream?

From your description, I expected to see some code _outside_ the "using
(ZipOutputStream...)" block that wrote the MemoryStream to the
"zipFileName" file, so that you could verify the stream as a valid ZIP
file using some independent tool (like Windows' built-in ZIP file
support). But instead, I see the opposite.

Not only was I surprised not to see what I expected, I'm not really
clear on what the code I do see is supposed to accomplish. It seems,
well...a little random to me.

Pete

 

[Guest]

I tried the call to flush() with no difference. The last item is still not
being added to the list.

foreach (string file in filenames)
{
ZipEntry entry = new ZipEntry(file);
entry.DateTime = DateTime.Now;
s.PutNextEntry(entry);
// TODO: last file is not added as entry
byte[] ba = GetBlob(13);
s.Write(ba, 0, ba.Length);
s.Flush()
}

I agree the code is a bit confusing. My objective is to read a byte array,
which represents one file, from from the database BLOB field; which is
working fine...then add each one to a zip file (ultimately in memory so I can
insert the zip file as a byte[] back to the blob field). However, for
testing reasons instead of doing the insert I am trying to write the zip file
to disk, which all this code is doing is creating an empty zip container.

So what i need help with is
1. figuring out why the last item is not added to the list
2. most importantly, why the zip file that is written to disk is empty.

Sorry for the confusion.

And one more thing...

[...]
byte[] buffer = new byte[4096];
foreach (string file in filenames)
{
ZipEntry entry = new ZipEntry(file);
entry.DateTime = DateTime.Now;
s.PutNextEntry(entry);
// TODO: last file is not added as entry
byte[] ba = GetBlob(13);
s.Write(ba, 0, ba.Length);
}

This part seems okay, at least without knowing anything specific about
the class, and keeping in mind the possibility that you need to call
some sort of flush method.

But this part confuses me:

// TODO: write stream to BLOB
//...But before I try that test that the zip is valid by
writting the ZipOutputStream
// to disk
using (FileStream fs = File.OpenRead(zipFileName))
{
int sourceBytes;
do
{
sourceBytes = fs.Read(buffer, 0, buffer.Length);
s.Write(buffer, 0, sourceBytes);
} while (sourceBytes > 0);
}

What exactly are you trying to do here? What is the contents of the
"test.zip" file? What is the point of appending those contents to your
ZipOutputStream? And why does it make sense to append those contents to
the last ZipEntry in the ZipOutputStream?

From your description, I expected to see some code _outside_ the "using
(ZipOutputStream...)" block that wrote the MemoryStream to the
"zipFileName" file, so that you could verify the stream as a valid ZIP
file using some independent tool (like Windows' built-in ZIP file
support). But instead, I see the opposite.

Not only was I surprised not to see what I expected, I'm not really
clear on what the code I do see is supposed to accomplish. It seems,
well...a little random to me.

Pete

 

[Peter Duniho]

[...]
So what i need help with is
1. figuring out why the last item is not added to the list

I don't know the answer to this. You might want to contact the author
of the class you're using. However, the fact that you could potentially
write additional junk to the ZipOutputStream at the end might corrupt
that last entry, causing it to not be seen later. I wouldn't think a
zero-length file would cause this problem, but I suppose you never know.
Depends on how well the ZipOutputStream handles zero-length writes (it
should handle them fine, but all code has bugs).

2. most importantly, why the zip file that is written to disk is empty.

You haven't posted any code that writes to a disk file, so it's not
really possible to answer this question. Though, if you've posted all
of the code you have, then the lack of any code that would do that is
likely the answer itself.

You should look more closely at the section of the code that I described
as confusing. It reads from a file and writes the data to the
ZipOutputStream, which seems to be the opposite of what you say you want.

Pete

 

[Guest]

Attached is my code that i cleaned up a bit. My previous issues still exist,
however, I am focused at figuring out why the zip file is corrupt in cases #1
and #2. Case #2 should be easy to recreate, just change the path to point to
a pdf on your system. I've added comments to assist you along the way.

using System;
using System.Collections.Generic;
using System.Text;
using System.IO;
using System.Data.SqlClient;
using System.Configuration;
using System.Data;
using ICSharpCode.SharpZipLib;
using ICSharpCode.SharpZipLib.Zip;

namespace testzipper
{
class Program
{
static void Main(string[] args)
{

//string[] files = new string[] { "c:\\temp\\test1.txt",
"c:\\temp\\test2.txt", "c:\\temp\\test5.pdf", "c:\\temp\\test6.doc" };
StreamZipper("test.zip");
}

// This method outputs a zip file to a memorystream
// The input in all three test cases is a byte array from different
sources
// (This will create a zip file with 1 file)
private static void StreamZipper(string zipFileName) // zipFileName
is not used, it is hardcoded for testing
{
try
{
// test case #1: read a BLOB (from the database) into a byte
array to create zip file
// the byte array is fine, however the zip file created is
corrupt.
// this byte array is valid when output as a file type
//byte[] ba = GetBlob(13);
// end test case #1

// test case #2: read the valid blob above from it's PDF
FILE (from disk) directly into the byte[]
// again...the zip file created is corrupt
// the pdf is valid
//FileInfo fInfo = new FileInfo("c:\\temp\\test5.pdf");
//long numBytes = fInfo.Length;
//FileStream fStream = new FileStream("c:\\temp\\test5.pdf",
FileMode.Open, FileAccess.Read);
//BinaryReader br = new BinaryReader(fStream);
//byte[] ba = br.ReadBytes((int)numBytes);
//string len = Convert.ToString(ba.Length);
//br.Close();
//fStream.Close();
// end test case #2

// test case #3: create a byte array from a string then
create a zip file
// this works, the zip file is valid..not sure why it shows
the zip file size as 4 million KB...but it works
byte[] ba = Encoding.Default.GetBytes("TEST");
// end test case #3

// create the zip file in memory
System.IO.MemoryStream ms = new MemoryStream();
using (ZipOutputStream s = new ZipOutputStream(ms))
{

s.SetLevel(9); // 0 - store only to 9 - means best
compression
//byte[] buffer = new byte[4096];

// you'll need to switch these depending on the case#
you are running...sorry
ZipEntry entry = new ZipEntry("file1.txt"); // for test
case #3
//ZipEntry entry = new ZipEntry("file1.pdf"); // for
test case #1 and #2
entry.DateTime = DateTime.Now;
s.PutNextEntry(entry);
s.Write(ba, 0, ba.Length);
s.Finish();
s.Close();
}

// ms is the memory stream that contains the zip file that I
will insert into a BLOB.
// let's test that the ms is valid by writting it out to a
zip file first.
byte[] zipms = (byte[])ms.ToArray();
using (BinaryWriter binWriter = new
BinaryWriter(File.Open(@"C:\\temp\\test.zip", FileMode.Create)))
{
//byte[] bb = new byte[] { 65, 66, 67 };

// write the zip file to disk from the memorystream
binWriter.Write(zipms);
binWriter.Write(ba);
}

}
catch (Exception ex)
{
Console.WriteLine("Exception during processing {0}", ex);

}

}

private static byte[] GetBlob(int id)
{
byte[] data = null;
string fileName = string.Empty;
string cn = "Server=localhost\\SQLEXPRESS;Initial
Catalog=XXX;user id=xxx; Password=xxx;";
using (SqlConnection mySqlConnection = new SqlConnection(cn))
{
SqlCommand myCommand = new SqlCommand("SPNAME1",
mySqlConnection);
myCommand.CommandType = CommandType.StoredProcedure;

SqlParameter prmId = new SqlParameter("@ID", SqlDbType.Int);
prmId.Value = id;
myCommand.Parameters.Add(prmId);

mySqlConnection.Open();
using (SqlDataReader myReader = myCommand.ExecuteReader())
{
if (myReader.Read())
{
//
data =
(byte[])myReader[myReader.GetOrdinal("PDFData")];
fileName = (string)myReader["FileName"];
}
myReader.Close();
}
mySqlConnection.Close();
}
return data;
}
}
}

[...]
So what i need help with is
1. figuring out why the last item is not added to the list

I don't know the answer to this. You might want to contact the author
of the class you're using. However, the fact that you could potentially
write additional junk to the ZipOutputStream at the end might corrupt
that last entry, causing it to not be seen later. I wouldn't think a
zero-length file would cause this problem, but I suppose you never know.
Depends on how well the ZipOutputStream handles zero-length writes (it
should handle them fine, but all code has bugs).

2. most importantly, why the zip file that is written to disk is empty.

You haven't posted any code that writes to a disk file, so it's not
really possible to answer this question. Though, if you've posted all
of the code you have, then the lack of any code that would do that is
likely the answer itself.

You should look more closely at the section of the code that I described
as confusing. It reads from a file and writes the data to the
ZipOutputStream, which seems to be the opposite of what you say you want.

Pete