StreamReader poor performance

[oafyuf]

Hi,
I'm having performanbce issues with StreamReader and was wondering
what I could do to improve it...

The following takes around 3 seconds to process! The content of the
response is:

"<?xml version="1.0" ?><ERROR>ORA-01403: no data found</ERROR>"

HttpWebRequest req = (HttpWebRequest)WebRequest.Create(strURIQuery);
req.Method = "GET";
req.Credentials = new NetworkCredential(strUser, strPass);
StreamReader stream = new
StreamReader(req.GetResponse().GetResponseStream());
StringBuilder sbXMLResponse0 = new StringBuilder();

//## start timer here
while((strLineXMLResponse = stream.ReadLine()) != null)
{
if(strLineXMLResponse.Length > 0 )
{
sbXMLResponse0.Append(strLineXMLResponse);
}
}
//## end timer here


I also tried:
strLineXMLResponse = stream.ReadToEnd();
with similar processing times. The response times are sub-second - I
think I have correctly isolated the performance issue to the
StreamReader.

Any help would be much appreciated.
Thanks,
Oafyuf

 

[Jon Skeet [C# MVP]]

I'm having performanbce issues with StreamReader and was wondering
what I could do to improve it...

The following takes around 3 seconds to process! The content of the
response is:

"<?xml version="1.0" ?><ERROR>ORA-01403: no data found</ERROR>"

HttpWebRequest req = (HttpWebRequest)WebRequest.Create(strURIQuery);
req.Method = "GET";
req.Credentials = new NetworkCredential(strUser, strPass);
StreamReader stream = new
StreamReader(req.GetResponse().GetResponseStream());
StringBuilder sbXMLResponse0 = new StringBuilder();

//## start timer here
while((strLineXMLResponse = stream.ReadLine()) != null)
{
if(strLineXMLResponse.Length > 0 )
{
sbXMLResponse0.Append(strLineXMLResponse);
}
}
//## end timer here


I also tried:
strLineXMLResponse = stream.ReadToEnd();
with similar processing times. The response times are sub-second - I
think I have correctly isolated the performance issue to the
StreamReader.

Try reading straight from the response stream - I suspect it will take
about the same length of time. I'd be very surprised if it really was
StreamReader being slow.

 

[Ignacio Machin \( .NET/ C# MVP \)]

Hi,

How is your connection with the server? what about the time to process the
request?
maybe there is the bottleneck, and not in the length of the response.
StreamReader should not add any overhead.

Cheers,

 

[John Wood]

Are you sure it's not the GetResponseStream that's taking so long? I mean,
that function call performs a network trip to the host specified in the URI,
which of course will take a considerable amount of time.

My experience with StreamReader so far have not shown any significant
performance problems.

 

[oafyuf]

Thanks for all your responses. Sorry it takes me so long to reply but
the site I'm on at the moment does not allow newsserver access and I'm
doing this through Google...

I understand that StreamReader is regarded as fast, but the 3 seconds
average timing I'm getting is between my "start timer here" and "end
timer here" comments. I have another timer block around the HTTP GET
which is returning a response sub-second. The network connection is
currently on a dev LAN with little other traffic. I should probably
mention that I'm doing URI Queries on an Oracle 9i database (getting
XML back in the response). Perhaps there is a way to allocate the size
of the StreamReader before it reads the stream or use another kind of
buffering?

I suppose it's not the StringBuilder because I still get the same
timing with this:

while((strLineXMLResponse = stream.ReadLine()) != null)
// do nothing
}

and this:

strLineXMLResponse = stream.ReadToEnd();

It seems the issue may not be with StreamReader, but it appears to be
in reading the stream into a string. I believe I *must* do this before
I can load it in an XmlDocument, but is there a way of loading the
stream directly into the XmlDocument?

Is the XmlTextReader more appropriate?

Thanks,
Oafyuf

 

[Jon Skeet [C# MVP]]

It seems the issue may not be with StreamReader, but it appears to be
in reading the stream into a string. I believe I *must* do this before
I can load it in an XmlDocument, but is there a way of loading the
stream directly into the XmlDocument?

Well, you could use XmlDocument.Load(Stream) but I doubt that it will
be significantly faster.

I suggest, just for the sake of testing, that you read the entire
contents of the stream into a MemoryStream, reset the position and
*then* use StreamReader - I suspect it'll show the StreamReader being
fast, and the reading from the network to be slow.

Could you post a short but complete program which demonstrates the
problem?

See http://www.pobox.com/~skeet/csharp/complete.html for details of
what I mean by that.

 

[oafyuf]

Hi Jon,

The following times about the same (3 seconds):

XmlTextReader reader = new
XmlTextReader(req.GetResponse().GetResponseStream())

.... which doesn't distinguish between the HTTP response time and the
load into XmlTextReader but maybe it supports your proposition?
However, pasting the URI Query directly into a browser returns data in
around 1.5 seconds. Anyhow, I *think* I am doing my timing after the
response has been completed, unless:

StreamReader stream = new
StreamReader(req.GetResponse().GetResponseStream());

doesn't do the request when you declare/assign it but when it is read.

Thanks,
Oafyuf

 

[Jon Skeet [C# MVP]]

The following times about the same (3 seconds):

XmlTextReader reader = new
XmlTextReader(req.GetResponse().GetResponseStream())

... which doesn't distinguish between the HTTP response time and the
load into XmlTextReader but maybe it supports your proposition?

Not really. Chances are XmlTextReader is using the same kind of code as
StringReader, so if one is genuinely slow, the other is likely to be as
well.

However, pasting the URI Query directly into a browser returns data in
around 1.5 seconds. Anyhow, I *think* I am doing my timing after the
response has been completed, unless:

StreamReader stream = new
StreamReader(req.GetResponse().GetResponseStream());

doesn't do the request when you declare/assign it but when it is read.

Just because you've got the response stream doesn't mean it's read all
the data, necessarily, does it? (Not sure how much it buffers, if any,
to be honest.)

Just try timing the read from the stream until its end - see how long
that takes.

 

[oafyuf]

Sorry All,
You are all correct, it is the response time that is slow. I tried:

XmlTextReader reader = new XmlTextReader("http://myURL/data.xml");

.... which is extremely fast, as opposed to:

XmlTextReader reader = new
XmlTextReader(req.GetResponse().GetResponseStream());

.... which is tediously slow. I just can't understand why I was getting
the timings in the part of the code which does the read instead of the
HTTP response. It's just as well that I'm not on the test team

Oafyuf

 

[oafyuf]

Thanks Jon,
I tried using a MemoryStream and it confirmed my other posting in that
it is the response that is the problem - it was just that my original
diagnosis was faulty. So... I'm off to an Oracle NG to find out if I
can improve the performance of URI queries. Thanks for your help!
Oafyuf