multi-threading web requests

[Guest]

Hi

I'd like to know whether it's beneficial to multi-thread web request calls

Experimentally, it doesn't look very helpful. I measured the time it takes to make request to a certain website from a C# application, and doing it serially 10 consequtive calls took about 20 seconds (2 seconds each, average)

Then, I tried spawning off 10 different threads, each making a single request. The first thread finished in 2 seconds, the second one in 4 seconds, etc. The total time came out to be around 16-17 seconds. While I expected all the calls to complete in vicinity of 3-5 seconds

Does that mean, that an underlaying protocol still serializes all my requests? Is is the web server that I am dealing processes everything serially? What am I missing

This is how I access a web site

WebRequest oRequest = WebRequest.Create(szUrl)
WebResponse oResponse = oRequest.GetResponse()

Thanks
VR

 

[Alvin Bruney [MVP]]

what it probably means is that spawning 10 threads is expensive enough to
make the venture futile. Microsoft recommends 15 threads tops. I usually
stay way below 10 except if i am sure numbers above that will be in short
burst processing.

Modify your time to only include the request/response and eliminate the
thread expense. You may see better results.

--
Regards,
Alvin Bruney [ASP.NET MVP]
Got tidbits? Get it here...
http://tinyurl.com/3he3b

Hi.

I'd like to know whether it's beneficial to multi-thread web request calls.

Experimentally, it doesn't look very helpful. I measured the time it takes

to make request to a certain website from a C# application, and doing it
serially 10 consequtive calls took about 20 seconds (2 seconds each,
average).

Then, I tried spawning off 10 different threads, each making a single

request. The first thread finished in 2 seconds, the second one in 4
seconds, etc. The total time came out to be around 16-17 seconds. While I
expected all the calls to complete in vicinity of 3-5 seconds.

Does that mean, that an underlaying protocol still serializes all my

requests? Is is the web server that I am dealing processes everything
serially? What am I missing?

 

[Guest]

Alvin

Thanks for the reply. I should have mentioned that I did try the same approach with a fewer threads, and I got same results -- gain in time was insignificant and the data looked like something was serializing my requests

I will definitely try to make modifications you suggested. I just wanted to make sure that there isn't something in the .NET / Windows architecture that you might know, that could somehow serialize all HTTP calls

Thanks

V



----- Alvin Bruney [MVP] wrote: ----

what it probably means is that spawning 10 threads is expensive enough t
make the venture futile. Microsoft recommends 15 threads tops. I usuall
stay way below 10 except if i am sure numbers above that will be in shor
burst processing

Modify your time to only include the request/response and eliminate th
thread expense. You may see better results

--
Regards
Alvin Bruney [ASP.NET MVP
Got tidbits? Get it here..
http://tinyurl.com/3he3

Hi

to make request to a certain website from a C# application, and doing i
serially 10 consequtive calls took about 20 seconds (2 seconds each
average)request. The first thread finished in 2 seconds, the second one in
seconds, etc. The total time came out to be around 16-17 seconds. While
expected all the calls to complete in vicinity of 3-5 secondsrequests? Is is the web server that I am dealing processes everythin
serially? What am I missing

 

[David Browne]

Hi.

I'd like to know whether it's beneficial to multi-thread web request calls.

Experimentally, it doesn't look very helpful. I measured the time it takes

to make request to a certain website from a C# application, and doing it
serially 10 consequtive calls took about 20 seconds (2 seconds each,
average).

Then, I tried spawning off 10 different threads, each making a single

request. The first thread finished in 2 seconds, the second one in 4
seconds, etc. The total time came out to be around 16-17 seconds. While I
expected all the calls to complete in vicinity of 3-5 seconds.

Does that mean, that an underlaying protocol still serializes all my

requests? Is is the web server that I am dealing processes everything
serially? What am I missing?
By default the WebRequest will only allow 2 simultaneous requests to the
same host. This behavior is inspired by how browsers like IE behave. But
it's not very desirable for some situations like web services. This setting
is completely configurable, however, see

http://msdn.microsoft.com/library/d...ref/html/gngrfconnectionmanagementelement.asp

David

 

[Guest]

Thanks, David.

I did tried changing the settings in machine.config and it worked. Now I am getting my responses back simulataneosly.

I noticed a strange thing though: a link you specified said to have the following format
<connectionManagement><add name = "www.contoso.com" maxconnection = "4" /><add name = "*" maxconnection = "2" /></connectionManagement

whereas, machine.config required "address" instead of "name". Is that an older version of .NET mentioned in the link? The one I have installed on my PC is v1.1.4322

Thanks again for your help
V

----- David Browne wrote: ----

Hi

to make request to a certain website from a C# application, and doing i
serially 10 consequtive calls took about 20 seconds (2 seconds each
average)request. The first thread finished in 2 seconds, the second one in
seconds, etc. The total time came out to be around 16-17 seconds. While
expected all the calls to complete in vicinity of 3-5 secondsrequests? Is is the web server that I am dealing processes everythin
serially? What am I missing


By default the WebRequest will only allow 2 simultaneous requests to th
same host. This behavior is inspired by how browsers like IE behave. Bu
it's not very desirable for some situations like web services. This settin
is completely configurable, however, se

http://msdn.microsoft.com/library/d...nref/html/gngrfconnectionmanagementelement.as

Davi

 

[Jeffrey Tan[MSFT]]

Hi VR,

I am glad you got what you want.

For you another concern, yes, this is a known document incorrect in our
MSDN sample. Please ignore the incorrect sample.

Thank you for your patience and cooperation. If you have any questions or
concerns, please feel free to post it in the group. I am standing by to be
of assistance.

Best regards,
Jeffrey Tan
Microsoft Online Partner Support
Get Secure! - www.microsoft.com/security
This posting is provided "as is" with no warranties and confers no rights.

 

[Guest]

Thanks, Jeffrey

V

----- \"Jeffrey Tan[MSFT]\" wrote: ----

Hi VR

I am glad you got what you want

For you another concern, yes, this is a known document incorrect in our
MSDN sample. Please ignore the incorrect sample

Thank you for your patience and cooperation. If you have any questions or
concerns, please feel free to post it in the group. I am standing by to be
of assistance

Best regards
Jeffrey Ta
Microsoft Online Partner Suppor
Get Secure! - www.microsoft.com/securit
This posting is provided "as is" with no warranties and confers no rights