PC Review


Reply
Thread Tools Rate Thread

Configuring More Than Two HttpWebRequests To The Same Host?

 
 
Alexander J. Oss
Guest
Posts: n/a
 
      13th Jun 2006
I'm writing a C# application for .NET 2.0 in which I intend a number of
background worker threads to be conveying information from our SQL Server
database to a legacy database via HTTP GET requests. Each worker thread
runs the following code in a loop:

-----
HttpWebRequest request = (HttpWebRequest)WebRequest.Create([snip]);

HttpWebResponse response = (HttpWebResponse)request.GetResponse();

Stream responseStream = response.GetResponseStream();
StreamReader responseReader = new StreamReader(responseStream);
string responseText = String.Empty;
string line;
while ((line = responseReader.ReadLine()) != null)
{
if (line != String.Empty)
responseText += line;
}

responseReader.Close();
response.Close();
-----

The total number of requests will end up being approximately 30,000 for each
run of the transfer, so I'm trying to run as many threads as I can so the
total run time will be reduced. But it appears that .NET by default
restricts the number of simultaneous network connections to a single host to
2 (why?!). I did find that if I create an application configuration file
that looks like this:

<?xml version="1.0" encoding="utf-8" ?>
<configuration>
<system.net>
<connectionManagement>
<add address = "10.134.0.2" maxconnection = "24" />
<add address = "*" maxconnection = "2" />
</connectionManagement>
</system.net>
</configuration>

I should be able to have up to 24 connections to that 10.134.0.2 host (see
http://support.microsoft.com/kb/821268/en-us).

However, this isn't working--I still seem to be constrained to two
connections. Any suggestions as to what I might be doing wrong? The
application name is RightsTransferToADAMS.exe and I have a
RightsTransferToADAMS.exe.config file in the same directory with the content
above. Is there a way to programmatically check that the config file has
been loaded correctly and that I should be able to make 24 connections? Do
I need to do something else?

Thanks very much for any help you might be able to provide.


 
Reply With Quote
 
 
 
 
Greg Young
Guest
Posts: n/a
 
      13th Jun 2006
Take a look at the ServicePoint class
http://msdn2.microsoft.com/en-us/sys...vicepoint.aspx

http://msdn2.microsoft.com/en-us/library/7af54za5.aspx includes a bit more
on it.

Cheers,

Greg Young
MVP - C#
http://codebetter.com/blogs/gregyoung

"Alexander J. Oss" <(E-Mail Removed)> wrote in message
news:%(E-Mail Removed)...
> I'm writing a C# application for .NET 2.0 in which I intend a number of
> background worker threads to be conveying information from our SQL Server
> database to a legacy database via HTTP GET requests. Each worker thread
> runs the following code in a loop:
>
> -----
> HttpWebRequest request = (HttpWebRequest)WebRequest.Create([snip]);
>
> HttpWebResponse response = (HttpWebResponse)request.GetResponse();
>
> Stream responseStream = response.GetResponseStream();
> StreamReader responseReader = new StreamReader(responseStream);
> string responseText = String.Empty;
> string line;
> while ((line = responseReader.ReadLine()) != null)
> {
> if (line != String.Empty)
> responseText += line;
> }
>
> responseReader.Close();
> response.Close();
> -----
>
> The total number of requests will end up being approximately 30,000 for
> each run of the transfer, so I'm trying to run as many threads as I can so
> the total run time will be reduced. But it appears that .NET by default
> restricts the number of simultaneous network connections to a single host
> to 2 (why?!). I did find that if I create an application configuration
> file that looks like this:
>
> <?xml version="1.0" encoding="utf-8" ?>
> <configuration>
> <system.net>
> <connectionManagement>
> <add address = "10.134.0.2" maxconnection = "24" />
> <add address = "*" maxconnection = "2" />
> </connectionManagement>
> </system.net>
> </configuration>
>
> I should be able to have up to 24 connections to that 10.134.0.2 host (see
> http://support.microsoft.com/kb/821268/en-us).
>
> However, this isn't working--I still seem to be constrained to two
> connections. Any suggestions as to what I might be doing wrong? The
> application name is RightsTransferToADAMS.exe and I have a
> RightsTransferToADAMS.exe.config file in the same directory with the
> content above. Is there a way to programmatically check that the config
> file has been loaded correctly and that I should be able to make 24
> connections? Do I need to do something else?
>
> Thanks very much for any help you might be able to provide.
>



 
Reply With Quote
 
=?Utf-8?B?UGV0ZXIgQnJvbWJlcmcgW0MjIE1WUF0=?=
Guest
Posts: n/a
 
      13th Jun 2006
Alexander,
If you use a ThreadPool and the asynchronous pattern with async callback
methods, you should be able to make as many connections as you need (within
reason).
Peter

--
Co-founder, Eggheadcafe.com developer portal:
http://www.eggheadcafe.com
UnBlog:
http://petesbloggerama.blogspot.com




"Alexander J. Oss" wrote:

> I'm writing a C# application for .NET 2.0 in which I intend a number of
> background worker threads to be conveying information from our SQL Server
> database to a legacy database via HTTP GET requests. Each worker thread
> runs the following code in a loop:
>
> -----
> HttpWebRequest request = (HttpWebRequest)WebRequest.Create([snip]);
>
> HttpWebResponse response = (HttpWebResponse)request.GetResponse();
>
> Stream responseStream = response.GetResponseStream();
> StreamReader responseReader = new StreamReader(responseStream);
> string responseText = String.Empty;
> string line;
> while ((line = responseReader.ReadLine()) != null)
> {
> if (line != String.Empty)
> responseText += line;
> }
>
> responseReader.Close();
> response.Close();
> -----
>
> The total number of requests will end up being approximately 30,000 for each
> run of the transfer, so I'm trying to run as many threads as I can so the
> total run time will be reduced. But it appears that .NET by default
> restricts the number of simultaneous network connections to a single host to
> 2 (why?!). I did find that if I create an application configuration file
> that looks like this:
>
> <?xml version="1.0" encoding="utf-8" ?>
> <configuration>
> <system.net>
> <connectionManagement>
> <add address = "10.134.0.2" maxconnection = "24" />
> <add address = "*" maxconnection = "2" />
> </connectionManagement>
> </system.net>
> </configuration>
>
> I should be able to have up to 24 connections to that 10.134.0.2 host (see
> http://support.microsoft.com/kb/821268/en-us).
>
> However, this isn't working--I still seem to be constrained to two
> connections. Any suggestions as to what I might be doing wrong? The
> application name is RightsTransferToADAMS.exe and I have a
> RightsTransferToADAMS.exe.config file in the same directory with the content
> above. Is there a way to programmatically check that the config file has
> been loaded correctly and that I should be able to make 24 connections? Do
> I need to do something else?
>
> Thanks very much for any help you might be able to provide.
>
>
>

 
Reply With Quote
 
Alexander J. Oss
Guest
Posts: n/a
 
      14th Jun 2006
Okay, here's what I ended up doing. I'm sure there's redundancy in there,
but I'm not taking the time to experiment with which things are necessary or
not at the moment. Besides... it turns out that with only a few threads
(5-10) I can max out our old HP 9000 D270's two CPUs (it's a CGI app that
kicks off a Progress database client process with each call... horribly
inefficient).

1. Created the application configuration file I had in the earlier post.

2. Added this line of code before creating any HttpWebRequest objects:

ServicePointManager.DefaultConnectionLimit = 24;

3. Added these lines of code for each HttpWebRequest object created:

request.KeepAlive = false;
request.ServicePoint.ConnectionLimit = 24;
request.ProtocolVersion = HttpVersion.Version10;

Thanks again for everyone's help!

"Vadym Stetsyak" <(E-Mail Removed)> wrote in message
news:(E-Mail Removed)...
> Hello, Alexander!
>
> <skip>
>
> AJO> The total number of requests will end up being approximately 30,000
> AJO> for each run of the transfer, so I'm trying to run as many threads as
> AJO> I can so the total run time will be reduced. But it appears that
> .NET
> AJO> by default restricts the number of simultaneous network connections
> to
> AJO> a single host to 2 (why?!). I did find that if I create an
> AJO> application configuration file that looks like this:
>
> AFAIR 2 connections is the number recommended connection stated in the
> HTTP RFC
>
> The question how to do more connections was discussed earlier in this or
> related group
> look at (
> http://groups.google.com.ua/group/mi...221216c41f0a5b
> )
>
> (
> http://groups.google.com.ua/group/mi...cbd5801911ae53 )
>
> --
> Regards, Vadym Stetsyak
> www: http://vadmyst.blogspot.com



 
Reply With Quote
 
 
 
Reply

Thread Tools
Rate This Thread
Rate This Thread:

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Using Multiple HttpWebRequests jperezvazquez@gmail.com Microsoft C# .NET 4 28th Oct 2007 07:57 PM
HttpWebRequests and Redirect Gina_Marano Microsoft C# .NET 5 18th May 2007 06:50 PM
Configuring More Than Two HttpWebRequests To The Same Host? Alexander J. Oss Microsoft Dot NET 3 14th Jun 2006 04:51 AM
Inserting Cookies into HttpWebRequests Pat Allan Microsoft C# .NET 0 12th Jan 2004 12:59 PM
Re: Getting HttpWebRequests Joerg Jooss Microsoft C# .NET 0 12th Jul 2003 12:09 PM


Features
 

Advertising
 

Newsgroups
 


All times are GMT +1. The time now is 06:50 AM.