Is there a clean way to create a byte[] from an HttpWebRequest.Read?

[John Wilmot]

The following code-snippet works but is just not elegant, IMO.

Does anyone have a recommendation to make this more efficient?

For example, I prefer NOT to have a static "buf" size. In fact, I would
prefer something similar to StreamReader.ReadToEnd() for returning a string,
but into a byte[] array.

Snippet:

HttpWebResponse webResponse = (HttpWebResponse) webRequest.GetResponse();
using (BinaryReader reader = new
BinaryReader(webResponse.GetResponseStream(), enc))
{
byte[] buf = new byte[20000];
int count = reader.Read(buf, 0, buf.Length);
byte[] resp = new byte[count];
for (int i = 0; i < count; i++) resp = buf;
}

 

[Marc Gravell]

Snippet is unreliable; assumes < 20K bytes, and manual copy (instead of
buf.CopyTo() or Buffer.BlockCopy()) is inneficient. Perhaps read (in a
loop, using a smaller buffer) from the response and writing to a
MemoryStream, then call ms.ToArray()? This would work with any size.

In 2.0, you could instead use WebClient:

byte[] data;
using (WebClient wc = new WebClient()) {
data = wc.DownloadData("http://somewhere");
}

Marc

 

[John Wilmot]

The thing I don't like is having to preallocate a set buffer size.

OTH, I actually started by using webclient but it turned out that it did not
have enough granularity for setting things like timeout values, etc.

 

[Marc Gravell]

Then don't! Use the MemoryStream approach I mentioned... although you should
probably cap the size to some arbitrary value to avoid it trying to allocate
a huge array... it uses a small, disposable buffer when copying between the
streams; no need to allocate your own over-sized array anywhere... Not sure
if you need the encoding / BinaryReader here, since we are reading binary
anyway... I've left it in so as not to break anything...

(snippet not tested, but looks sound)

Marc

HttpWebResponse webResponse =
(HttpWebResponse)webRequest.GetResponse();
byte[] data;
const int BUFFER_SIZE = 2048;
int bytesRead;
byte[] buffer = new byte[BUFFER_SIZE];
using (MemoryStream ms = new MemoryStream())
using (BinaryReader reader = new
BinaryReader(webResponse.GetResponseStream(), enc)) {
while ((bytesRead = reader.Read(buffer, 0, BUFFER_SIZE)) > 0) {
ms.Write(buffer, 0, bytesRead);
}
data = ms.ToArray();
}

 

[Joerg Jooss]

Thus wrote John,

The thing I don't like is having to preallocate a set buffer size.

OTH, I actually started by using webclient but it turned out that it
did not have enough granularity for setting things like timeout
values, etc.

Note that in .NET 2.0, you can subclass WebClient and override GetWebRequest(),
which gives you full control over the underlying HttpWebRequest.

Cheers,

 

[Marc Gravell]

Thinking about it, you could always allocate an array sized using the
content-length header...? If the data is a different length, you probably
have other issues, so throw...?

Marc

 

[Joerg Jooss]

Thus wrote Marc,

Thinking about it, you could always allocate an array sized using the
content-length header...?

[...]

Not reliably. There are situations in which there is no Content-Length header.

Cheers,

 

[Marc Gravell]

I knew I should have just kept with the mem-stream suggestion. Me and my big
mouth...

Marc