Need for efficient method for File Parsing and copying

[Tom]

I am trying to update another developers code, and am stuck in a
cludge.

It works like this:

A user uploads a file from the web page. Our code reads from a
NetworkStream to a Filestream, and writes out a temporary file. Then
the file is parsed (The file contains HTML data + the actual binary
file that was transmitted). As part of the parsing, once the main
binary file is to be parsed out, it is written with a new Filestream,
byte by byte to a new location on disk.

This was great when the file was 24Mb, but now it is 70Mb, and our
windows CE device can not hold 2 copies of this large file. Not to
mention that the additional copy takes a while to perform.

To me it seams that there are two methods to solve this, the first
being to read the networkstream realtime until the actual binary file
shows up, then filestream directly. The second method, which to me
seems simpler, is to just take the large temp file, and truncate off
the non-binary data.

By the way, I can not use a memorystream, as I have alocated most of
the device memory to storage, thus there is not enough memory room.

Anyone have any advice??

Tom Kuhn

 

[Ignacio Machin ( .NET/ C# MVP )]

A user uploads a file from the web page.

Upload or download?
From the description below it seems that you are getting the file from
a webserver to a Win CE device.

 Our code reads from a
NetworkStream to a Filestream, and writes out a temporary file.  Then
the file is parsed (The file contains HTML data + the actual binary
file that was transmitted).  As part of the parsing, once the main
binary file is to be parsed out, it is written with a new Filestream,
byte by byte to a new location on disk.



This was great when the file was 24Mb, but now it is 70Mb, and our
windows CE device can not hold 2 copies of this large file.  Not to
mention that the additional copy takes a while to perform.

You can delete the old file. If this is not possible cause you need at
least one working copy. then you can try to zip it . download the new
file (it should also be zipped) and only when the new file is
downloaded ok then y ou delete the old one and unzip the new one.

Note that with files that big it might take a LONG time to download &
process all the info. Maybe it's time to change the delivery method of
your app.

Anyone have any advice??

As I said above, maybe it's time to rethink your strategic. 70MB only
ni data is a LOT especially when some devices have only 128MB

 

[Tom]

Zipping... Now thats an idea that for some reason never crossed my
mind! The file that we are copying is a full Win CE Flash file system
image and is 63Mb unzipped... Well I zipped it and WOW, only 12Mb.
Since we alreeady handle unzipping in the device, it should be no
problem to go this route. Good Idea.

As for what we were doing, the file was transfered via HTTP POST from
a users computer. The networkStream grabbed to POST data and built a
temp file with the file along with the post data in it. I was trying
to real0time analyse the data and find the beginning of the actual
file in a byte by byte process reading from the stream.

I think this zip solution will work out well.

Thanks,
Tom

 

[Cheeso]

I think thiszipsolution will work out well.

Thanks,
Tom

How do you do zipping / unzipping on the device?