Scalability of asynchronous sockets?

[taskswap]

I have a legacy application written in C that I'm trying to convert to
C#. It processes a very large amount of data from many clients
(actually, upstream servers - this is a mux) simultaneously.

I've read through what must be dozens of ways to do socket
communication in C#, and it seems they all devolve into three basic
options - Socket.Select, IOCP through a native interface, and
Asynchronous callbacks. I'm fine using Asynchronous callbacks, which
all the documentation and articles I've read seem to indicate is the
most scalable solution. My concern is over the ThreadPool involved.

I might have 40-60 connections talking simultaneously, and during peak
periods that can be 150+. Note that this ran fine without threads by
handling the data in round-robin fashion, so it can handle blocking if
that's what will happen when the thread pool gets starved. What it
can't tolerate is ADDITIONAL performance degradation. That is, if
hitting that magic 25-thread limit is going to make things not just N
ticks slower, but N^2 ticks slower as the system struggles to do some
thread magic I don't know about, then I'll have issues.

So, some questions:
What, exactly, will happen when I starve the ThreadPool?

Is there a list somewhere of other I/O (like file) activity I'll affect
by doing this?

If I have a dual-proc + HT box, 4 virtual processors, does this mean MY
application gets 100 theoretical asynch threads to use (25 per
processor)? Scalability via CPU is just fine with me.

Will I affect other applications running on the box (is the ThreadPool
shared across all of .Net, or just my application?)

Should I be trying to write a custom ThreadPool? If so, are there any
pointers on how to tell socket asynchronous functions to use the custom
pool? Or do I just make real threads and use blocking I/O?

Is there housekeeping overhead while the sockets are asleep? (If I have
3000 clients connected but very few talking, is that a problem?)

Do I really need a StateObject class? I have a wrapper class that does
all the dirty work described above. Any reason I can't just keep the
byte[] buffer and other elements in the wrapper and use "this" when I
call BeginReceive?

Any reason I can't just reference my instance variables without even
bothering with the StateObject indirection? I was going to instantiate
a new wrapper for every connection.

Thanks,
Chad

 

[Vadym Stetsyak]

What, exactly, will happen when I starve the ThreadPool?
WorkItems that you've queued into it will not receive thread until thread
pool threads will become free.
Additionaly, how do you perform your network I/O?
ThreadPool has 2 types of threads : WorkerThreads and CompletionPort threads

AFAIK BeginSend/BeginReceive asycn socket methods use CompletionPort
threads, and their number
in Thread Pool is large.

Watch for word wrap
(
http://msdn.microsoft.com/library/default.asp?url=/library/en-us/dndotnet/html/progthrepool.asp )
( http://msdn.microsoft.com/msdnmag/issues/05/08/HighPerformanceSockets/ )

Is there a list somewhere of other I/O (like file) activity I'll affect
by doing this?

If server most of the time will be completing your I/O ( processor and
memory usage will be high )
then it may affect other I/O types or simply downgrage whole server
performance.

Will I affect other applications running on the box (is the ThreadPool
shared across all of .Net, or just my application?)

ThreadPool is static that is it operates on the AppDomain. Every AppDomain
has it own ThreadPool

Should I be trying to write a custom ThreadPool?

Only if current implementation doesn't suit your needs well.

If so, are there any pointers on how to tell socket asynchronous functions
to use the custom
pool? Or do I just make real threads and use blocking I/O?

AFAIK ThreadPool is used internally if you're using
Socket.BeginReceive/BeginSend async methods

Is there housekeeping overhead while the sockets are asleep? (If I have
3000 clients connected but very few talking, is that a problem?)

No it is not a problem, or at least it has not to be a problem. However, it
is more efficient to keep
the connection number not too large ( every connection allocates system
resources - memory for instance ).

Do I really need a StateObject class? I have a wrapper class that does
all the dirty work described above. Any reason I can't just keep the
byte[] buffer and other elements in the wrapper and use "this" when I
call BeginReceive?

Then you will have to provide one wrapper for each connection.

--
Vadym Stetsyak aka Vadmyst
http://vadmyst.blogspot.com

I have a legacy application written in C that I'm trying to convert to
C#. It processes a very large amount of data from many clients
(actually, upstream servers - this is a mux) simultaneously.

I've read through what must be dozens of ways to do socket
communication in C#, and it seems they all devolve into three basic
options - Socket.Select, IOCP through a native interface, and
Asynchronous callbacks. I'm fine using Asynchronous callbacks, which
all the documentation and articles I've read seem to indicate is the
most scalable solution. My concern is over the ThreadPool involved.

I might have 40-60 connections talking simultaneously, and during peak
periods that can be 150+. Note that this ran fine without threads by
handling the data in round-robin fashion, so it can handle blocking if
that's what will happen when the thread pool gets starved. What it
can't tolerate is ADDITIONAL performance degradation. That is, if
hitting that magic 25-thread limit is going to make things not just N
ticks slower, but N^2 ticks slower as the system struggles to do some
thread magic I don't know about, then I'll have issues.

So, some questions:
What, exactly, will happen when I starve the ThreadPool?

Is there a list somewhere of other I/O (like file) activity I'll affect
by doing this?

If I have a dual-proc + HT box, 4 virtual processors, does this mean MY
application gets 100 theoretical asynch threads to use (25 per
processor)? Scalability via CPU is just fine with me.

Will I affect other applications running on the box (is the ThreadPool
shared across all of .Net, or just my application?)

Should I be trying to write a custom ThreadPool? If so, are there any
pointers on how to tell socket asynchronous functions to use the custom
pool? Or do I just make real threads and use blocking I/O?

Is there housekeeping overhead while the sockets are asleep? (If I have
3000 clients connected but very few talking, is that a problem?)

Do I really need a StateObject class? I have a wrapper class that does
all the dirty work described above. Any reason I can't just keep the
byte[] buffer and other elements in the wrapper and use "this" when I
call BeginReceive?

Any reason I can't just reference my instance variables without even
bothering with the StateObject indirection? I was going to instantiate
a new wrapper for every connection.

Thanks,
Chad

 

[Willy Denoyette [MVP]]

The IOC threadpool is 1000 on v1.1 and v2.0.

Willy.

|> What, exactly, will happen when I starve the ThreadPool?
| WorkItems that you've queued into it will not receive thread until thread
| pool threads will become free.
| Additionaly, how do you perform your network I/O?
| ThreadPool has 2 types of threads : WorkerThreads and CompletionPort
threads
|
| AFAIK BeginSend/BeginReceive asycn socket methods use CompletionPort
| threads, and their number
| in Thread Pool is large.
|
| Watch for word wrap
| (
|
http://msdn.microsoft.com/library/default.asp?url=/library/en-us/dndotnet/html/progthrepool.asp )
| ( http://msdn.microsoft.com/msdnmag/issues/05/08/HighPerformanceSockets/ )
|
|
| > Is there a list somewhere of other I/O (like file) activity I'll affect
| > by doing this?
|
| If server most of the time will be completing your I/O ( processor and
| memory usage will be high )
| then it may affect other I/O types or simply downgrage whole server
| performance.
|
| > Will I affect other applications running on the box (is the ThreadPool
| > shared across all of .Net, or just my application?)
|
| ThreadPool is static that is it operates on the AppDomain. Every AppDomain
| has it own ThreadPool
|
| > Should I be trying to write a custom ThreadPool?
| Only if current implementation doesn't suit your needs well.
|
| > If so, are there any pointers on how to tell socket asynchronous
functions
| > to use the custom
| > pool? Or do I just make real threads and use blocking I/O?
|
| AFAIK ThreadPool is used internally if you're using
| Socket.BeginReceive/BeginSend async methods
|
| > Is there housekeeping overhead while the sockets are asleep? (If I have
| > 3000 clients connected but very few talking, is that a problem?)
| No it is not a problem, or at least it has not to be a problem. However,
it
| is more efficient to keep
| the connection number not too large ( every connection allocates system
| resources - memory for instance ).
|
| > Do I really need a StateObject class? I have a wrapper class that does
| > all the dirty work described above. Any reason I can't just keep the
| > byte[] buffer and other elements in the wrapper and use "this" when I
| > call BeginReceive?
|
| Then you will have to provide one wrapper for each connection.
|
| --
| Vadym Stetsyak aka Vadmyst
| http://vadmyst.blogspot.com
|
| | >I have a legacy application written in C that I'm trying to convert to
| > C#. It processes a very large amount of data from many clients
| > (actually, upstream servers - this is a mux) simultaneously.
| >
| > I've read through what must be dozens of ways to do socket
| > communication in C#, and it seems they all devolve into three basic
| > options - Socket.Select, IOCP through a native interface, and
| > Asynchronous callbacks. I'm fine using Asynchronous callbacks, which
| > all the documentation and articles I've read seem to indicate is the
| > most scalable solution. My concern is over the ThreadPool involved.
| >
| > I might have 40-60 connections talking simultaneously, and during peak
| > periods that can be 150+. Note that this ran fine without threads by
| > handling the data in round-robin fashion, so it can handle blocking if
| > that's what will happen when the thread pool gets starved. What it
| > can't tolerate is ADDITIONAL performance degradation. That is, if
| > hitting that magic 25-thread limit is going to make things not just N
| > ticks slower, but N^2 ticks slower as the system struggles to do some
| > thread magic I don't know about, then I'll have issues.
| >
| > So, some questions:
| > What, exactly, will happen when I starve the ThreadPool?
| >
| > Is there a list somewhere of other I/O (like file) activity I'll affect
| > by doing this?
| >
| > If I have a dual-proc + HT box, 4 virtual processors, does this mean MY
| > application gets 100 theoretical asynch threads to use (25 per
| > processor)? Scalability via CPU is just fine with me.
| >
| > Will I affect other applications running on the box (is the ThreadPool
| > shared across all of .Net, or just my application?)
| >
| > Should I be trying to write a custom ThreadPool? If so, are there any
| > pointers on how to tell socket asynchronous functions to use the custom
| > pool? Or do I just make real threads and use blocking I/O?
| >
| > Is there housekeeping overhead while the sockets are asleep? (If I have
| > 3000 clients connected but very few talking, is that a problem?)
| >
| > Do I really need a StateObject class? I have a wrapper class that does
| > all the dirty work described above. Any reason I can't just keep the
| > byte[] buffer and other elements in the wrapper and use "this" when I
| > call BeginReceive?
| >
| > Any reason I can't just reference my instance variables without even
| > bothering with the StateObject indirection? I was going to instantiate
| > a new wrapper for every connection.
| >
| > Thanks,
| > Chad
| >
|
|

 

[taskswap]

OK, thanks for the tips. One more question. What is the behavior of
BeginReceive/EndReceive if I close the socket? Are they guaranteed not
to be called? The documentation makes it appear as though I can't
cancel the request in any way.

This is mainly a concern for transmissions. I want to make a queue and
do asynchronous sends. If I close the connection, though, I want to
wipe out the queue. The trouble is, I may (will) have already called
BeginSend and given the address of one of these buffers. If I then
release the buffer, I'm concerned that something will crash somewhere.

Also, I was hoping to preallocate an "empty" list of pending queue
nodes to increase performance. If necessary I can retain the object I
passed to EndReceive somewhere but that gets tricky.

So, what will happen to pending BeginReceive/EndReceive events if the
socket is disconnected, either by the remote, or by me calling
Shutdown() and/or Close()?

Thanks,
Chad

 

[Vadym Stetsyak]

So, what will happen to pending BeginReceive/EndReceive events if the

socket is disconnected, either by the remote, or by me calling
Shutdown() and/or Close()?

EndReceive/EndSend will return either exception or SocketError ( .NET 2.0 )
The error or exception will have errornumber. if connected was closed
gracefully
EndReceive will return 0 without exception or error.