You can exhaust the threadpool, of course, but that's when items start
to get queued.
That's not what Chris is talking about. He's referring to a design bug in
which code executing in a thread pool thread waits for another thread
executing in a thread pool thread to complete. IMHO, no one should ever
write code like this anyway, but if they do they can create a situation
where all of the thread pool threads are stuck waiting on other threads
(typically threads they created, but that's not important) that have all
been queued and won't execute until a currently running thread pool thread
finishes its task.
Well, isn't that the reason why the threadpool should have a sensible
maximum number of threads? To keep the scheduling overhead sensible and
avoid too much concurrency. That's why the built-in threadpools size
depends on the number of CPUs/cores, isn't it?
As I wrote in my other post, once you have more than twice as many
constantly executing threads than CPUs, there's no additional overhead
from scheduling. The only real overhead there is the context switch, and
once all threads are taking their entire timeslice, you don't get any more
context switch overhead with more threads than you would have with that
baseline number.
With more threads comes the greater possibility of repeatedly voiding the
CPU cache of useful data, but again even a small number of threads can do
that. I don't think that the possibility is a significant factor in the
choice of the number of threads in a thread pool.
Instead, I think that the number of threads in a thread pool is primarily
dictated by two balancing factors: the cost of a thread, and the need to
ensure that you always have some thread runnable, to maximize your use of
the CPU. When your tasks are entirely CPU-bound, this is less of an
issue: just make one thread for each CPU and let it run. But many
problems are at least partially i/o-bound. In that case, you want enough
threads to ensure that when some of those threads wind up waiting on i/o,
there are other threads that can be executed taking advantage of the
available CPU.
On the other side of the balance is the cost of a thread. With 1MB of
stack space for a thread (by default), and a 2GB limit in a processes
virtual address space, you can have only 2048 threads total, in theory (of
course, in practice it's even less than that because other things compete
for the process address space). Without this cost, you would just let the
thread pool grow unbounded, because after all that would maximize your
chances of always having something to run.
Well, you'd have around 25-50 threads on a typical thread pool on a
single/dual-core computer. If you have more tasks, they'll get queued
up.
But as Chris points out, that's not true any more. As of the first
service pack for .NET 2.0, the maximum number of active threads in the
default thread pool is 250 per CPU.
I believe that was his point, or at least part of it. I agree that with
the original limit of 25, it's not a problem to throw a bunch of tasks at
the thread pool. But unfortunately the Microsoft folks decided to work
around badly designed applications by raising the limit to 250 from 25.
This creates a significant risk of consuming process address space for a
few applications that assume the limit is lower, have a very large number
of tasks, and use the thread pool for their task queuing.
I guess the lesson here (or at least one of them) is that if you're going
to use a thread pool to handle your task queuing, include code to make
sure that the maximum thread count of threads for the thread pool is
something reasonable.
Pete
