Is there a clean way to create a byte[] from an HttpWebRequest.Read?

  • Thread starter Thread starter John Wilmot
  • Start date Start date
J

John Wilmot

The following code-snippet works but is just not elegant, IMO.

Does anyone have a recommendation to make this more efficient?

For example, I prefer NOT to have a static "buf" size. In fact, I would
prefer something similar to StreamReader.ReadToEnd() for returning a string,
but into a byte[] array.

Snippet:

HttpWebResponse webResponse = (HttpWebResponse) webRequest.GetResponse();
using (BinaryReader reader = new
BinaryReader(webResponse.GetResponseStream(), enc))
{
byte[] buf = new byte[20000];
int count = reader.Read(buf, 0, buf.Length);
byte[] resp = new byte[count];
for (int i = 0; i < count; i++) resp = buf;
}
 
Snippet is unreliable; assumes < 20K bytes, and manual copy (instead of
buf.CopyTo() or Buffer.BlockCopy()) is inneficient. Perhaps read (in a
loop, using a smaller buffer) from the response and writing to a
MemoryStream, then call ms.ToArray()? This would work with any size.

In 2.0, you could instead use WebClient:

byte[] data;
using (WebClient wc = new WebClient()) {
data = wc.DownloadData("http://somewhere");
}

Marc
 
The thing I don't like is having to preallocate a set buffer size.

OTH, I actually started by using webclient but it turned out that it did not
have enough granularity for setting things like timeout values, etc.
 
Then don't! Use the MemoryStream approach I mentioned... although you should
probably cap the size to some arbitrary value to avoid it trying to allocate
a huge array... it uses a small, disposable buffer when copying between the
streams; no need to allocate your own over-sized array anywhere... Not sure
if you need the encoding / BinaryReader here, since we are reading binary
anyway... I've left it in so as not to break anything...

(snippet not tested, but looks sound)

Marc

HttpWebResponse webResponse =
(HttpWebResponse)webRequest.GetResponse();
byte[] data;
const int BUFFER_SIZE = 2048;
int bytesRead;
byte[] buffer = new byte[BUFFER_SIZE];
using (MemoryStream ms = new MemoryStream())
using (BinaryReader reader = new
BinaryReader(webResponse.GetResponseStream(), enc)) {
while ((bytesRead = reader.Read(buffer, 0, BUFFER_SIZE)) > 0) {
ms.Write(buffer, 0, bytesRead);
}
data = ms.ToArray();
}
 
Thus wrote John,
The thing I don't like is having to preallocate a set buffer size.

OTH, I actually started by using webclient but it turned out that it
did not have enough granularity for setting things like timeout
values, etc.
Click to expand...

Note that in .NET 2.0, you can subclass WebClient and override GetWebRequest(),
which gives you full control over the underlying HttpWebRequest.

Cheers,
 
Thinking about it, you could always allocate an array sized using the
content-length header...? If the data is a different length, you probably
have other issues, so throw...?

Marc
 
Thus wrote Marc,
Thinking about it, you could always allocate an array sized using the
content-length header...?
Click to expand...
[...]

Not reliably. There are situations in which there is no Content-Length header.

Cheers,
 
I knew I should have just kept with the mem-stream suggestion. Me and my big
mouth...

Marc
 
Back
Top