Custom Thread Pool in c#

[Venkat]

Hi,
I am working on an application (developed using c#2.0) which needs to do a
big job and when you start the job in a single thread it takes long time to
complete. So, we want to break the job and run in multiple threads. I heard
about .Net Thread pool class but it has some limitations like we can't have
more than 25 threads in it and read some articles which explain the problems
with the Standard thread pool in case any of thread throws an exception and
UI problems.
Many people suggested writing custom Thread pool to solve my problem. But I
am not good at threading and I wonder whether there is any Custom thread
pool which runs robust and gains performance with multi core processors.
Basically, in my application I need to have ability to cancel any of thread
in the thread pool. Can anyone suggest me links/source code for a customized
thread pool?

Thanks in advance.

 

[Peter Duniho]

Hi,
I am working on an application (developed using c#2.0) which needs to
do a big job and when you start the job in a single thread it takes
long time to complete. So, we want to break the job and run in multiple
threads. I heard about .Net Thread pool class but it has some
limitations like we can't have more than 25 threads in it and read some
articles which explain the problems with the Standard thread pool in
case any of thread throws an exception and UI problems.
Many people suggested writing custom Thread pool to solve my problem.
But I am not good at threading and I wonder whether there is any Custom
thread pool which runs robust and gains performance with multi core
processors. Basically, in my application I need to have ability to
cancel any of thread in the thread pool. Can anyone suggest me
links/source code for a customized thread pool?

IMHO, you answered the question when you wrote "I am not good at
threading". If that's true, you should try as hard as possible to
stick with the built-in implementations.

You should be aware that for CPU-bound tasks, having more threads
running than you have CPU cores is only going to _reduce_ performance.
Even for i/o-bound tasks, there will be a point of diminishing returns.
I really doubt that you actually need more than 25 active threads, and
if your task is CPU-bound then even running as many as 25 threads is
going to be counter-productive (unless you've got yourself a 32-core
system ).

You're not very specific about what "problems with the Standard thread
pool" concern you, but with respect to the things you do mention:

* Can't have more than 25 threads. As I mentioned above, that's
not likely to be an actual problem.

* Threads throwing an exception. If you care about detecting
exceptions, it should be simple enough to catch them in your worker
thread entry point and deal with them there.

* UI problems. Assuming this is in regards to the prohibition
against cross-thread access to GUI components, you're going to run into
this issue no matter how you implement your threading. It's a
threading issue, not something specific to the ThreadPool class.

* Canceling individual threads. As with the exceptions, this is
something you should handle within the worker thread code, and you
would handle it the same whether you use ThreadPool or your own custom
implementation.

Finally, consider that if and when you run into problems using a thread
pool implementation, you will have a lot more success receiving help in
this newsgroup if you're using an implementation others are familiar
with. In other words, the built-in .NET implementation rather than
some custom implementation (whether yours or someone else's).

Pete

 

[donna.gravell]

With regards to cancelling... this is always much better if done
gracefully (perhaps via a flag on each item) than forcefully. If you
can design your code to support this it would be much better. A
(synchronised) "cancelling" bool would suffice (similar to the
BackgroundWorker approach).

At the core, a custom thread-pool is just a worker thread (or threads)
and a synchronised delegate queue (or similar). Jon Skeet has a usable
sample here:
http://www.pobox.com/~skeet/csharp/miscutil/
It should do much of what you want, or provide a guide for writing
your own.

Marc

 

[Marc Gravell]

(oops; weekend over; I'll revert to "Marc", then...)