Why Would ResponseStream().Read return zero bytes when not at end offile?

[Rymfax]

Greetings,

I have an app that uses HttpWebRequest to download a rather large file
(100 MB). I have a while loop set up so that it reads 4096 bytes at a
time from the ResponseStream() until it reads zero bytes. The odd
thing is that for some of the people using my app, the Read() function
from the ResponseStream() will return zero bytes even though it has
not read the enter ResponseStream(). I know it hasn't read the entire
stream because I do a ResponseStream().ContentLength and get the size
of the stream (and record it) before I start reading data, then I also
record how many bytes I've read.

Any idea why this would be happening? And any suggestions on how to
get around?

 

[Peter Duniho]

Greetings,

I have an app that uses HttpWebRequest to download a rather large file
(100 MB). I have a while loop set up so that it reads 4096 bytes at a
time from the ResponseStream() until it reads zero bytes. The odd
thing is that for some of the people using my app, the Read() function
from the ResponseStream() will return zero bytes even though it has
not read the enter ResponseStream(). I know it hasn't read the entire
stream because I do a ResponseStream().ContentLength and get the size
of the stream (and record it) before I start reading data, then I also
record how many bytes I've read.

Any idea why this would be happening? And any suggestions on how to
get around?

Under what circumstances does this happen?

Assuming the request is on a TCP connection (which is the usual
mechanism), you'll get a zero byte read any time the connection has been
closed. In the normal case, this happens when the data's all been sent
and you've read everything there is to read. But I suppose you could have
an HTTP server that's closing the connection prematurely or something like
that. The ContentLength is just what the HTTP server said it would send;
there's no guarantee when you get that value that there is actually that
much data ready and waiting for you to receive.

It may be difficult to diagnose the problem if you can't come up with a
reproducible scenario. If you can, then the first thing I would do is run
a network analysis program (like Wireshark, for example) and see what's
really going on with the connection. If the connection is in fact being
closed before you've read all the data (which seems to me to be the most
likely explanation), then you may be dealing with a server issue (or
possibly even an ISP issue...after the whole Comcast/BitTorrent fiasco, I
have little faith that all ISPs are leaving the data stream alone all the
time).

Before you do all that, you may of course want to just double-check your
code and make sure that your logic is simple and doesn't have any way that
it might accidently read something without including it in the final data
count. For example, some sort of odd threading implementation where data
can be lost due to synchronization problems between threads. I think a
person would have to go to some effort to write code like that, but I've
seen worse. It could happen, and you should double-check and make sure
that's not going on here.

Pete

 

[Arne Vajhøj]

I have an app that uses HttpWebRequest to download a rather large file
(100 MB). I have a while loop set up so that it reads 4096 bytes at a
time from the ResponseStream() until it reads zero bytes. The odd
thing is that for some of the people using my app, the Read() function
from the ResponseStream() will return zero bytes even though it has
not read the enter ResponseStream(). I know it hasn't read the entire
stream because I do a ResponseStream().ContentLength and get the size
of the stream (and record it) before I start reading data, then I also
record how many bytes I've read.

Any idea why this would be happening? And any suggestions on how to
get around?

It sounds weird.

The docs are rather clear:

#Return Value
#The total number of bytes read into the buffer. This can be less than
#the number of bytes requested if that many bytes are not currently
#available, or zero (0) if the end of the stream has been reached.

Is the file in question public available so we can try ?

Arne

 

[Marc Gravell]

I don't know why it is dropping ,but if the web-server supports it,
you could attempt to resume by adding a RANGE header to the request
and re-sending? (i.e. from byte {large number} onwards...)

Marc

 

[Bob Powell [MVP]]

Would the amount that you actually can download correspond to about four
megabytes?

--
--
Bob Powell [MVP]
Visual C#, System.Drawing

Ramuseco Limited .NET consulting
http://www.ramuseco.com

Find great Windows Forms articles in Windows Forms Tips and Tricks
http://www.bobpowell.net/tipstricks.htm

Answer those GDI+ questions with the GDI+ FAQ
http://www.bobpowell.net/faqmain.htm

All new articles provide code in C# and VB.NET.
Subscribe to the RSS feeds provided and never miss a new article.

 

[Rymfax]

Under what circumstances does this happen?

Assuming the request is on a TCP connection (which is the usual  
mechanism), you'll get a zero byte read any time the connection has been  
closed.  In the normal case, this happens when the data's all been sent  
and you've read everything there is to read.  But I suppose you could have  
an HTTP server that's closing the connection prematurely or something like 
that.  The ContentLength is just what the HTTP server said it would send;  
there's no guarantee when you get that value that there is actually that  
much data ready and waiting for you to receive.

It may be difficult to diagnose the problem if you can't come up with a  
reproducible scenario.  If you can, then the first thing I would do is run  
a network analysis program (like Wireshark, for example) and see what's  
really going on with the connection.  If the connection is in fact being 
closed before you've read all the data (which seems to me to be the most  
likely explanation), then you may be dealing with a server issue (or  
possibly even an ISP issue...after the whole Comcast/BitTorrent fiasco, I  
have little faith that all ISPs are leaving the data stream alone all the  
time).

Before you do all that, you may of course want to just double-check your  
code and make sure that your logic is simple and doesn't have any way that 
it might accidently read something without including it in the final data  
count.  For example, some sort of odd threading implementation where data  
can be lost due to synchronization problems between threads.  I think a  
person would have to go to some effort to write code like that, but I've  
seen worse.  It could happen, and you should double-check and make sure  
that's not going on here.

Pete

Pete,

You bring up some good points, thanks. I did re-check my code and
there is no way for it to exit the loop except to press cancel (which
sets a flag to true and is checked in the loop) or for zero bytes to
be read from the ResponseStream. FYI, this is the resposne stream
from HttpWebRequest.

It is possible that it is reading zero bytes because the connection is
closing prematurely, but this is suppose to generate an exception
which I check for (I actually check for any exception). So I don't
know what could be causing the ResponseStream to return zero bytes if
it hasn't reached the end of the content (according to the
ContentLength anyway). I've already coded in a re-check for when this
happens, but I was hoping to figure out WHY it is happening.

Anne,

Unfortunately the files aren't public, sorry.

Thanks for your help guys. IF you have any more insight, I'd be most
greateful.

 

[Peter Duniho]

[...]
It is possible that it is reading zero bytes because the connection is
closing prematurely, but this is suppose to generate an exception

That's not true. You'll get an exception if the connection is _reset_,
but if it's simply closed then you get a 0 byte receive.

which I check for (I actually check for any exception).
So I don't
know what could be causing the ResponseStream to return zero bytes if
it hasn't reached the end of the content (according to the
ContentLength anyway).

As I said, there is no network requirement that the number of bytes
delivered by the server before closing the connection is identical to the
number of bytes the server advertised in the HTTP response. The
advertised length is just that: an advertisement. The network side of
things doesn't enforce it.

One would expect a correctly-behaving HTTP server to provide the bytes as
advertised, but servers don't always behave, and on top of that some ISPs
are known to mess with TCP connections, including doing something like
closing it for no good reason.

I've already coded in a re-check for when this
happens, but I was hoping to figure out WHY it is happening.

That's an admirably goal. But the only way you can do that is by doing
some debugging. You need to monitor the network traffic at both ends and
figure out who is closing the connection. Once you've identified that,
then you can look at why. Unfortunately, a vague problem description is
not enough information for anyone else in this newsgroup to answer the
question. Only first-hand knowledge will do.

Pete