Async Socket: Rising Thread Count

[Morgan Leppink]

Hi all -

We have been developing a complex TCP socket communication
app that is responsible for keeping numerous connections
open to clients on routable IPs. The app receives request
on a non-routable listener socket that marshals
communications between the non-routable netowrk and the
clients out on the routable network.

The app runs as a .NET service in Win2K Server. Async
sockets are being used, with BeginSend(), BeginReceive(),
EndSend(), EndReceive(), etc. Please correct me if I'm
wrong, but our understanding is that these async events
execute on a thread provided by a thread pool created by
the OS.

Operationally, the app is doing exactly what it designed
to do, BUT, the total thread count for the application
gradually rises over time (as displayed by Task Manager).
It starts out at about 18 threads, but after 24 hours it's
up to over 60 threads, and it keeps rising from there.

As expected with an async app, the number of threads does
not have a direct relationship to the number of open
connections. However, it is the marshalling of numerous
TCP messages that seems to make the thread count rise.
The marshalling of messages is when those asynch events
fire, and so I assume that the OS is creating additional
threads from the pool when they are not neccesary for the
operation of the app.

I ALSO assume that we have a bug somewhere. What I am
looking for are likely causes of the OS continuing to
assign additional threads from the pool to my process -
for example, what types of cleanup failures would cause
this? What if EndSend() did not get called properly? The
app is huge, so I would like to narrow it down to the most
likely culprit(s) so we can track this down.

Any other sage advice regarding this type of app would
also be appreciated.

Thanks in advance,

Morgan Leppink
..

 

[Feroze [MSFT]]

In order to pinpoint what is happening, you will need to debug this
application. The best way is to attach a debugger of choice to the service,
and break in from time to time to see what the threads are doing. Are they
managed threads executing code in your application ? If so, that might give
you an idea of what code in your application is responsible.

On the other hand, if the Taskmanager is showing a lot of threads, but the
debugger does not show any managed code executing on them, it might be CLR
threads. Then you will have to do some native debugging (using windbg for
example) and see what each of those threads are doing.

Also, do you create threads explicitly, either through ThreadPoo.*
functions, or creating a new Thread object in CLR ?

--
Remove "user" from the email address to reply to the author.

This posting is provided "AS IS" with no warranties, and confers no rights

Use of included script samples are subject to the terms specified at
http://www.microsoft.com/info/cpyright.htm