A Scalable TCP Server

[SRLoka]

After reading the newsgroups and various .Net web sites, I have come to a
conclusion that BeginReceive and BeginSend are the way to go as they use
IOCompletion Ports(I have no clue what they mean but most C++ programmers
seem to prefer them, based on my readings). I have decided to use this
sample from MSDN as a starting point to build my TCP server.

http://msdn.microsoft.com/library/d...e/html/cpconUsingNon-BlockingServerSocket.asp

The server would eventually communicate with around 1500 to 2500 clients.
These clients try to keep the connection open as long as possible(they are
third party - over GPRS - and I have no control of it). So in theory I could
have all the clients connected at same time. Is the above suggested approach
good for this type of situation ? The server will parse the incoming data in
the readCallback function and perform database read/writes based on that and
send back data to the client.

Thanks in advance for any info

Srinivas

 

[William Stacey [MVP]]

Also see these refs.
http://www.devhood.com/tutorials/tutorial_details.aspx?tutorial_id=709
http://msdn.microsoft.com/library/d...html/cpconnon-blockingserversocketexample.asp

Async servers are good for many clients. They are harder to program
however. If your around <= ~1000 clients, then one or more threaded
server(s) may also be an option for you and much easier to program as
program flow is natural. If you truly will have 1500-2500 *active clients
really pounding, then you have to ask yourself if one machine will handle
it. Async is good, but if you really have that many active clients, your
machine will still need to share cpu between all the callbacks going on. If
not done right, you can also hit the wall on the thread pool as the IOCP on
done on iocp threadpool, but the callbacks are run on thread pool threads,
so you need to make sure you don't have a bunch of overlapped callbacks
going on (just something to watch out for.) Moreover, because the callback
are run on a thread pool thread, you want those methods as quick as
possible. There is another point where a threaded server can be better, as
one thread will (normally) not effect the other clients (each in its own
thread) and the os scheduler handles fairness between threads.

 

[SRLoka]

1) In fact the link I send is same code as your MSDN link. The one I send
has additional description it.
2) "If your around <= ~1000 clients, then one or more threaded
server(s) may also be an option for you and much easier to program as
program flow is natural."
I have actually a working server of this type. Right now only 5 test
clients. I got this code sample and idea on one of the C# sites too. Here a
single thread accepts incoming connections and passes each client to a
separate processing thread. To avoid too many threads and context switching,
this programmer(I cant remember his name) adds all connected sockets to a Q
and processes them in order. But my clients send data only every 3 to 10
minutes(configurable) but just stay connected. So looping through the Q is
an overkill.
3) I have gone through Asad Aziz's article(first link). Its a more refined
version of MSDN sample. I do not want to disconnect any socket even if there
is no data for a predefined time(I think he does that). I want to close it
only when the client disconnects. As per the docs, the BeginReceive will
throw an exception if the client disconnects and I want to catch this
exception and release that connection.

So is the server as written by Asad Aziz more scalable ? Like I said, the
clients send data very infrequently but they just stay connected. So does
that mean each BeginReceive will hold a IOCP thread ? What is the limit to
such threads ? Does it make a difference if its a beefy server ?

4)"but the callbacks are run on thread pool threads"
Are these thread pool limits per application or system wide ? If it is
per application, I can put the incoming data in MSMQ and another application
handle the parsing so that the Call Backs finish quickly.

Thank You

Srinivas

 

[William Stacey [MVP]]

I have actually a working server of this type. Right now only 5 test

clients. I got this code sample and idea on one of the C# sites too. Here a
single thread accepts incoming connections and passes each client to a
separate processing thread. To avoid too many threads and context switching,
this programmer(I cant remember his name) adds all connected sockets to a Q
and processes them in order.

Could you find the link for that please? I would be interested in looking
at that one. TIA.

3) I have gone through Asad Aziz's article(first link). Its a more refined
version of MSDN sample. I do not want to disconnect any socket even if there
is no data for a predefined time(I think he does that). I want to close it
only when the client disconnects. As per the docs, the BeginReceive will
throw an exception if the client disconnects and I want to catch this
exception and release that connection.

You could catch that with a blocking Receive also if you go the threaded
route.

So is the server as written by Asad Aziz more scalable ?

Not as far as I can tell. Similar to the MSDN one.

Like I said, the
clients send data very infrequently but they just stay connected. So does
that mean each BeginReceive will hold a IOCP thread ? What is the limit to
such threads ?

I think the IOCP thread pool is max 1000 threads. The default .net thread
pool that calls your callbacks is 25. This will queue your callbacks in
order. If one thread blocks, another thread is released from the pool up to
25 max. So keep your callbacks fast.

Are these thread pool limits per application or system wide ?

One per process.

per application, I can put the incoming data in MSMQ and another application
handle the parsing so that the Call Backs finish quickly.

Not sure about your app, but I don't think you need MSMQ for this and will
probably just get in the way.

 

[SRLoka]

Could you find the link for that please? I would be interested in looking

at that one. TIA.

I found it after going through my IE history
http://www.codeproject.com/dotnet/dotnettcp.asp
If you look at the source code, I think I used the server3 code.
See also the discussion under it - particularly the post by Anonymous
towards the end. This is when I started to research on IOCP and ended up
with the MSDN sample.

I think the IOCP thread pool is max 1000 threads.

Does that mean, if I have 1000 connections(some - actually most - may be
idle), further requests will be denied by the OS ?

Another approach I have seen is (just a description - I could not find any
samples) to add all the connected sockets to an IList and use the Select
Method.
Has anyone used this approach ?

So it looks like if the client is done sending data, it should disconnect or
the server should disconnect it after a timeout. Is that a correct
assumption ?

Thanks Again

Srinivas