Q: HttpWebResponse Transfer-Encoding chunked - how to read trailers?

  • Thread starter Thread starter Jacek
  • Start date Start date
J

Jacek

Hi!

My app does quite a bit of work downloading large chunks of data from
different web servers. One of problems I face is to provide approximate
progress reports to users showing download progress. While task is trivial
when web servers set Content-Length header in response its getting to be
really hard when server uses chunked encoding. The only way according to
HTTP 1.1 RFC 2616 to get info on total size of the chunked message is to
read optional TRAILER containing entity header fields (my solution anyway
would be optional).

The problem is that I see no ways to access chunked header or TRAILER in
..NET other than implementing my own HTTP classes on top of Net Sockets :(

Any ideas how to get it done easy way ?

cheers
 
But its not true since we are talking about internal chunked encoding
trailer headers which are not part of standard HTTP 1.1 headers. They are
embedded inside a chunk at its very end. Just check following link to get
closer idea what I mean about trailer in chunk:
http://www.w3.org/Protocols/rfc2616/rfc2616-sec3.html#sec3.6 and perhaps
appendix 19.4.6 of RFC 2616 to see pseudo code for parsing this data.

Simplest example of www pages served with chunked encoding are exotic eBay
sites i.e. China or India. Unfortunately changing protocol version to HTTP
1.0 does not solve problem (chunked is not supported there) since almost all
servers I work with despite dropping chunked encoding do not set
Content-Length header anyway.

Cheers
 
Hi Jacek,

thanks for you're note. I should be in the "Trailer" field of the
HttpResponseHeader. Did I mension that it's only .Net 2.0 ;)

Hope that helps,
Markus
 
Jacek said:
Hi!

My app does quite a bit of work downloading large chunks of data from
different web servers. One of problems I face is to provide
approximate progress reports to users showing download progress.
While task is trivial when web servers set Content-Length header in
response its getting to be really hard when server uses chunked
encoding. The only way according to HTTP 1.1 RFC 2616 to get info on
total size of the chunked message is to read optional TRAILER
containing entity header fields (my solution anyway would be
optional).

The problem is that I see no ways to access chunked header or TRAILER
in .NET other than implementing my own HTTP classes on top of Net
Sockets :(

Any ideas how to get it done easy way ?
Click to expand...

Cheat!

;-)

No really, when I look at the way Firefox seems to handle file
downloads with an unknown Content-Length, it's really nothing but
cheating: They use a progress bar which just scrolls through until the
end and then starts from zero again. This happens until the download is
finished.

This is hardly a progress bar in its original sense, but it shows that
something is going on...

Cheers,
 
Back
Top