Start Loading a file at an offset

[Chizl]

I'm trying to add 206 Partial content to my web server and I'm getting
errors each time I try using the offset from the Read command. Maybe I'm
not understanding the Read() command. I've scaled this demo back, but in
reality the file is 3.8GB, I need to send chunks of 100KB down to the
browser, but I need it to start with byte 106954752. The error doesn't make
sense to me, because it shouldn't be looking at my array it should be
looking at the file. The array should only be the storage. I even tried
adding the iStartBytes and iMaxSize together thinking maybe it's not wanting
the size, but where the end of the read should be. Same error.

The error is:
--------
Offset and length were out of bounds for the array or count is greater than
the number of elements from index to the end of the source collection.
--------

FileStream fs = new FileStream(sPhysicalFilePath, FileMode.Open,
FileAccess.Read, FileShare.Read);
BinaryReader reader = new BinaryReader(fs);

long lSize = fs.Length; //10MB
if(lSize>1048576) //no more than 1MB at a time
lSize=1048576;

byte[] bytes = new byte[lSize];

Int32 iMaxSize = bytes.Length; //1MB
Int32 iStartBytes = 2097152; //2MB Mark

//based on my params, I would assume I could start picking up
//the file at the 2MB mark and only get 1MB of data, but I get the error on
this line..
while ((read = reader.Read(bytes, iStartBytes, iMaxSize)) != 0)
{
.....
}

 

[Marc Gravell]

The offset param to Read relates to the buffer you are giving it, and
it almost always 0, i.e. start filling the buffer from the buffer's
start. It doesn't relate to the stream, which has a cursor (Position)
to it's own current location (starting at 0 for read).

What you want to do is to .Seek() the stream, or (clearer) change
the .Position to iStartBytes.

Marc

 

[Chizl]

Perfect.. Thanks.. I got it working with a 192MB Video using FDM..
However, when I tried the same thing with the 3.8GB Zip, it's as if FDM
quits never even talks to me.. Never sends me any headers or anything..

 

[Marc Gravell]

Well, I don't know exactly why without tracing it through, but I
suspect that 2Gb is the critical point. A good few things break at
this size, including many "zip" implementations.

But as long as you are just treating it as a byte-stream (which it
appears), then in *theory* it should work...

Have you tried a sniffer such as fiddler / ethereal to see what is
actually happening?

Marc

 

[Marc Gravell]

Another thought:

Int32 iMaxSize = bytes.Length; //1MB
Int32 iStartBytes = 2097152; //2MB Mark


If you are using Int32, but going about 2Gb, then there is a problem;
you'll need to use "long" / Int64 for the offset. Equally, 1Mb seems a
little large for a buffer. Not necessarily wrong, but unusual. Note
that when dealing with large streams you should be returning the data
via a loop over a smaller buffer, rather than attempting to read
everything you want in one read, then throwing it at the client in one
write.

Marc

 

[Marc Gravell]

Oops; just saw your "while", so ignore my loop comment - but my Int32
comment stands.

Marc

 

[Chizl]

--

Oops; just saw your "while", so ignore my loop comment - but my Int32
comment stands.

Marc

Thanks for the response.. Here is the problem. If I don't use an
offset, force zero in that param, I can download the file in full, all 3.8GB
to the browser. Meaning all the vars set appropriately. It's only when
I try to start with an offset does the 3.8GB break.

Now I did change the Read() to use ZERO always.. Before the read I am
doing:
fs.Position = lStartByte;
This is what fixed the problem where downloading a 192MB video was having
the problem. I use FDM to download in 5 parts at a time and it works
perfectly.

Byte[] has a 128MB limit, but even with that I have configs that set the var
while doing chunks, to never go over 50MB. Now I know 50MB is extremely
insane when thinking of a web server loading up 50MB at a time to send to a
browser, but I leave the option open for someone that may need it for
internal uses.

I've posted the Receive and Send Header log I create during this
transaction..
http://www.chizl.com/aspuploaded/trans.log

You can test the download yourself to see what I mean.. Keep in mind the
Zip file is not really a zip file, it's a text file with zeros in it, then
renamed to .zip..
http://kws.karland.com/downloads/WizardsInWinter.wmv - The 192MB video that
works fine..
http://kws.karland.com/downloads/large-test.zip - the 3.8GB zip of nothing
doesn't..

Don't use a program like FDM and you can download both with no problem..

 

[Chizl]

Oops; just saw your "while", so ignore my loop comment - but my Int32
comment stands.

Marc

Here is the strange part.. After FDM just stops, I can start it again by
clicking on resume in FDM.. It then resumes and works just fine to the
very end. I'm wondering now, if this could possibly be a FDM issue, not
my web server...