ThreadPool VS Thread

[Sin Jeong-hun]

Hi. I'm writing a Client/Multi-threaded Server program on Windows
Vista. It worked fine on Windows Vista, but when the server ran on
Windows XP,

I/O operation has been aborted because of either a thread exit or an
application request

exception randomly occurred at the OnReceive method (asynchronous tcp
stream reading). I searched all over the internet and found a post
posted few years ago. He had the same problem as me, and he said it
seemed like a bug of the .NET framework. He said changing normal
Thread to ThreadPool solved the problem but he couldn't explain the
reason. So I tried his explanation and it really solved the problem
(as least till now)

My original code was:
Dim client As TcpClient = Listener.AcceptTcpClient()
Dim t As New Thread(New ParameterizedThreadStart(AddressOf
OnNewClient))
t.Start(client)

And I rewrote like:
Dim client As TcpClient = Listener.AcceptTcpClient()
ThreadPool.QueueUserWorkItem(New WaitCallback(AddressOf OnNewClient),
client)

Could you please tell me what was wrong with my original code? And
using ThreadPool really solves the problem? I mean can there be any
other possiblilities of new problem?

Thank you as always.
--------------------------------------------------
PS, other parts of the code are
Private Sub OnNewClient(ByVal client As Object)
Interlocked.Increment(LastID)
If LastID = Integer.MaxValue Then
LastID = Integer.MinValue
End If
Dim c As TcpClient = CType(client, TcpClient)
Dim handler As New ClientHandler(c, LastID)
Clients.Add(handler)
AddHandler handler.MessageReceived, AddressOf OnMessageReceived
AddHandler handler.ConnectionClosed, AddressOf OnConnectionClosed
End Sub

Public Class ClientHandler
Public Sub New(ByVal client As TcpClient, ByVal id As Integer)
TheClient = client
_ClientID = id
ClientStream = client.GetStream()
Dim state As New StateObject()
state.WorkSocket = client.Client
client.Client.BeginReceive(state.RawBuffer, 0,
StateObject.BufferSize, SocketFlags.None, New AsyncCallback(AddressOf
OnReceive), state)
End Sub

 

[Mehdi]

I/O operation has been aborted because of either a thread exit or an
application request

exception randomly occurred at the OnReceive method (asynchronous tcp
stream reading). I searched all over the internet and found a post
posted few years ago. He had the same problem as me, and he said it
seemed like a bug of the .NET framework. He said changing normal
Thread to ThreadPool solved the problem but he couldn't explain the
reason. So I tried his explanation and it really solved the problem
(as least till now)

I've been bitten by this exact same problem a few days ago. Blame the
really poor documentation of the Socket classes. The problem is that "All
I/O initiated by a given thread is canceled when that thread exits. A
pending asynchronous operation can fail if the thread exits before the
operation completes." (taken from MSDN).

The thing is that this comment has only been added in the documentation of
..NET 2 but wasn't there as far as i can see in the doc of .NET 1 even
though .NET 1 suffers from the same problem.

So if you initate an asynchonous I/O operation in a thread (which is what
you do when you call BeginReceive in your ClientHandler class), this
operation will be canceled if this thread exits leading to the exception
you get in the callback method.

In your case, you manually create a thread, then in this thread you create
an instance of ClientHandler which calls BeginReceive after which your
thread has nothing to do anymore and therefore dies, canceling in the
process your asynchronous operation.

The solution is to always initate asynchronous I/O calls (that is call the
BeginXXX methods of I/O classes) from a thread that never dies (for the
duration of your application) or that you know won't die before the
operation has completed. Typically this would be either:

- the main UI thread in the case of a UI application
- a thread from the threadpool since these threads are always kept alive to
be re-used by the .NET runtime
- a thread from a custom thread pool that you know are kept alive for the
duration of your application
- an IOCP thread, which are the threads in which callbacks of asyncronous
I/O operations are executed. I can't say for sure that they are always kept
alive but so far i haven't had problems initiating async I/O operations
from an IOCP thread. It seems logical to be able to iniate a new async I/O
operation from the callback method of the previous one.

So your solution of using the threadpool works. Just beware of not
executing any long running code in a threadpool thread as they are in
limited numbers (25 / processor) and are used by the .NET framework
internally. Alternatively, if you've got a main thread in your application
that never dies, switch to using async I/O excusively so that you're always
working in IOCP threads.

 

[Sin Jeong-hun]

I've been bitten by this exact same problem a few days ago. Blame the
really poor documentation of the Socket classes. The problem is that "All
I/O initiated by a given thread is canceled when that thread exits. A
pending asynchronous operation can fail if the thread exits before the
operation completes." (taken from MSDN).

The thing is that this comment has only been added in the documentation of
.NET 2 but wasn't there as far as i can see in the doc of .NET 1 even
though .NET 1 suffers from the same problem.

So if you initate an asynchonous I/O operation in a thread (which is what
you do when you call BeginReceive in your ClientHandler class), this
operation will be canceled if this thread exits leading to the exception
you get in the callback method.

In your case, you manually create a thread, then in this thread you create
an instance of ClientHandler which calls BeginReceive after which your
thread has nothing to do anymore and therefore dies, canceling in the
process your asynchronous operation.

The solution is to always initate asynchronous I/O calls (that is call the
BeginXXX methods of I/O classes) from a thread that never dies (for the
duration of your application) or that you know won't die before the
operation has completed. Typically this would be either:

- the main UI thread in the case of a UI application
- a thread from the threadpool since these threads are always kept alive to
be re-used by the .NET runtime
- a thread from a custom thread pool that you know are kept alive for the
duration of your application
- an IOCP thread, which are the threads in which callbacks of asyncronous
I/O operations are executed. I can't say for sure that they are always kept
alive but so far i haven't had problems initiating async I/O operations
from an IOCP thread. It seems logical to be able to iniate a new async I/O
operation from the callback method of the previous one.

So your solution of using the threadpool works. Just beware of not
executing any long running code in a threadpool thread as they are in
limited numbers (25 / processor) and are used by the .NET framework
internally. Alternatively, if you've got a main thread in your application
that never dies, switch to using async I/O excusively so that you're always
working in IOCP threads.

Thank you so much. I didn't expect that I would get such a detailed
reply.
My server probably has to handle 50~100 clients at the same time
(max).
So if using ThreadPool can only create 25 threads, that could be a
problem.

You said that I have to initiate asynchronous reading in a thread that
never dies.
As you see my code, it's a multi-threaded server. If I initiate
reading clients' messages
other than the ClientHandler thread, how can I tell which client from
the other clients?

I'm sorry for my ignorance but I can't get the picture. Currently,
each ClientHandler
thread serves one client. If reading is initiated in other thread, how
can I keep the
ClientHandler running and make the ClientHandler process the client's
incoming data?

Was my approach for Multi-threaded server wrong at the first place? I
wonder if you
could tell me how to design the workflow of a multi-threaded server
that could handle
about 100 clients at most.

Thank you.

 

[Ben Voigt]

You said that I have to initiate asynchronous reading in a thread that

never dies.
As you see my code, it's a multi-threaded server. If I initiate
reading clients' messages
other than the ClientHandler thread, how can I tell which client from
the other clients?

Each delegate comes with a "this" pointer, so make the event handler a
method inside the class containing per-client data, then all member
variables will be accessible to the handler code as it runs.

I'm sorry for my ignorance but I can't get the picture. Currently,
each ClientHandler
thread serves one client. If reading is initiated in other thread, how
can I keep the
ClientHandler running and make the ClientHandler process the client's
incoming data?

Each client needs its own data structure, but not its own thread. There's
no reason to have one thread per client, in fact it's bad (per queueing
theory, the average response time is considerably lower with a single
high-rate processor, than many slow processors running in parallel, which is
what threading emulates).

Was my approach for Multi-threaded server wrong at the first place? I
wonder if you
could tell me how to design the workflow of a multi-threaded server
that could handle
about 100 clients at most.

Best performance is gotten with only one thread per processor. When you
receive a message, process it by sending the next outgoing message and
return to waiting for the next message. Note that messages in this case
aren't just client requests, typically a client request might trigger a
query to the database, <here you must return to the main listen loop> the
database's reply triggers the response message back to the client.

Only if you have to use a poorly designed library such as a blocking
database API, so you need extra threads to just sit and wait for that
operation to complete. With pure message passing, your thread never waits
as long as there is work to do for any client.

This architecture is sometimes called a state machine (by hardware engineers
like me), continuations (by computer sci folks familiar with Lisp and
Scheme), completion callbacks (by OS engineers), etc. Just some terms to
google for.

Ultimately, this is the same as the difference between sequential
command-line interfaces vs an event-driven GUI. And just as in that case,
you have to stop thinking in terms of "X must happen next" and start
thinking of "if data Y arrives, it enables me to finish preparing Z"

 

[William Stacey [C# MVP]]

| - a thread from the threadpool since these threads are always kept alive
to
| be re-used by the .NET runtime

Threads from thread pool and iocp pool can and do get collected after some
lag timeout if they are not used. So not sure this is safe either.

| - an IOCP thread, which are the threads in which callbacks of asyncronous
| I/O operations are executed. I can't say for sure that they are always
kept
| alive but so far i haven't had problems initiating async I/O operations
| from an IOCP thread. It seems logical to be able to iniate a new async I/O
| operation from the callback method of the previous one.

This seems to work and many code I see does this (and I have done it).
However, I also wonder if this is just a bug waiting to happen. Is it just
the fact we are getting lucky and the iocp thead is always keep alive during
our usage (as we normally get a reply pretty fast under normal conditions)?
Or does the os do some fixup behind the covers that allows the async call to
remain thread agnostic because we started it on an IOCP thread? I would
love to know the answer.

| So your solution of using the threadpool works. Just beware of not
| executing any long running code in a threadpool thread as they are in
| limited numbers (25 / processor) and are used by the .NET framework
| internally.

..Net 2.0 added api to set min and max threads on both the thread pool and
iocp pool.

| Alternatively, if you've got a main thread in your application
| that never dies, switch to using async I/O excusively so that you're
always
| working in IOCP threads.

Again back to second issue of calling Beginxxx on a iocp thread. It seems
to work fine, but wonder if we can force a failure. Will have to test this
sometime.
--wjs

 

[Mehdi]

This seems to work and many code I see does this (and I have done it).
However, I also wonder if this is just a bug waiting to happen. Is it just
the fact we are getting lucky and the iocp thead is always keep alive during
our usage (as we normally get a reply pretty fast under normal conditions)?
Or does the os do some fixup behind the covers that allows the async call to
remain thread agnostic because we started it on an IOCP thread? I would
love to know the answer.

I've done quite a lot of reading over the last few days regarding this
issue and it seems that nobody really knows the answer to that. There is an
awful lot of confusion when it comes to async socket operations. Everybody
seems to aggree that async socket calls should be prefered over sync calls
for scalabilty reasons but nobody seems to know how to actually use async
socket calls. And the documentation doesn't help either. I find this quite
astonishing as socket programming is as the heart of many applications
nowdays, not really a very specific, niche programming technique. How come
Microsoft doesn't come up with a white paper about these issues? For now,
i'll keep doing what everybody else does and hope for the best but i don't
feel particularly good about it either.

 

[William Stacey [C# MVP]]

| I've done quite a lot of reading over the last few days regarding this
| issue and it seems that nobody really knows the answer to that. There is
an
| awful lot of confusion when it comes to async socket operations. Everybody
| seems to aggree that async socket calls should be prefered over sync calls
| for scalabilty reasons but nobody seems to know how to actually use async
| socket calls. And the documentation doesn't help either. I find this quite
| astonishing as socket programming is as the heart of many applications
| nowdays, not really a very specific, niche programming technique. How come
| Microsoft doesn't come up with a white paper about these issues? For now,
| i'll keep doing what everybody else does and hope for the best but i don't
| feel particularly good about it either.

I agree. If it works as documented, there is a lot of broken async socket
code out there - they just don't know it.
--wjs

 

[Chris Mullins [MVP]]

I agree. This has me pretty worried, at some level.

On the other than, emperical evidence says things work pretty well, and
there's probably nothing to worry about. We've got thousands of hours
hammering on our stuff, and it's pretty darn reliable...