HttpWebResponse

[Fredo]

I'm using HttpWebRequest and HttpWebResponse to pull data from a web page.
Sometimes I don't get the complete content, but sometimes, even when this
happens, I get no exceptions either.

I thought I'd use the ContentLength property of the WebResponse to determine
if I've received all the data, but that always returns -1 (at least for the
pages I'm downloading).

What I'd like to do is re-queue the request if I don't get the complete
response, but I can't figure out how to be sure.

I do occasionally get WebExceptions for things like timeouts and other
problems, and I'm able to handle those fine. So I'm not sure why some of
these pages (maybe 1 in 30 or so) are coming back only partially. The site's
servers are heavily loaded which is why I don't always get the complete
response. If I load it up in a browser, I sometimes only get the partial
page as well, so browsers have the same problem, so I don't think it's my
code for reading the stream.

Thanks.

Fredo

 

[Nicholas Paldino [.NET/C# MVP]]

Fredo,

Unfortunately, you can't do anything about this. It is the server that
is having the problem, not you. Unless you know of some identifier in the
content of the page which indicates it is complete (which you can search
for), there is nothing that I can think of you could check for.

The reason that the ContentLength property returns -1 sometimes is
because the response does not send the Content-length header. Usually, for
dynamically-generated pages, it doesn't return this, since it can not know
the length of the content before it is generated.

Hope this helps.

 

[Fredo]

Nicholas,

Thanks for the information. Disappointing, but I guess I'll just have to
figure out some way of determining the integrity of the response.

Fredo

Fredo,

Unfortunately, you can't do anything about this. It is the server that
is having the problem, not you. Unless you know of some identifier in the
content of the page which indicates it is complete (which you can search
for), there is nothing that I can think of you could check for.

The reason that the ContentLength property returns -1 sometimes is
because the response does not send the Content-length header. Usually,
for dynamically-generated pages, it doesn't return this, since it can not
know the length of the content before it is generated.

Hope this helps.

--
- Nicholas Paldino [.NET/C# MVP]
- (e-mail address removed)

I'm using HttpWebRequest and HttpWebResponse to pull data from a web
page. Sometimes I don't get the complete content, but sometimes, even
when this happens, I get no exceptions either.

I thought I'd use the ContentLength property of the WebResponse to
determine if I've received all the data, but that always returns -1 (at
least for the pages I'm downloading).

What I'd like to do is re-queue the request if I don't get the complete
response, but I can't figure out how to be sure.

I do occasionally get WebExceptions for things like timeouts and other
problems, and I'm able to handle those fine. So I'm not sure why some of
these pages (maybe 1 in 30 or so) are coming back only partially. The
site's servers are heavily loaded which is why I don't always get the
complete response. If I load it up in a browser, I sometimes only get the
partial page as well, so browsers have the same problem, so I don't think
it's my code for reading the stream.

Thanks.

Fredo