Solution for running many threads

[mircu]

Hello,

What would be the best for implementing architecture that runs several,
independent, scheduled or run by the user on demand task in the same time?

Now in my program every task creates it's own thread and run in it and
after it completes the thread is destroyed automatically by the
framework. So it is the simplest solution and I think it can be done
better.

I thought about using ThreadPool but the limit of 25 thread can be a
limitation for me and also I read that thread pool is rather for short
time operations. But my tasks can be running from seconds to a few
hours. I quickly adopted ThreadPool method and I noticed that the
program do not takes so much CPU time as the my original implementation.
Are ThreadPool tasks run on lower priority?

Thanks in advance.

Regards,
mircu

 

[brett]

It sounds as though you have a good setup. If your threads have many
sleep states, the thread pool is usually more efficient here. Also,
look at using QueueUserWorkItem to queue threads for cases when there
aren't any available in the pool. You can set CorSetMaxThreads to
increase the number of available threads in the thread pool. However,
unless you really need all of those threads, I'd leave at the default
25.

The thread pool does a pretty good job of managing all your threads.
Are 25 threads not enough for your app? Using the threadpool limit
will keep your app from draining computer resources and allow other
apps to run more smoothly.

Brett

 

[mircu]

It sounds as though you have a good setup. If your threads have many
sleep states, the thread pool is usually more efficient here. Also,
look at using QueueUserWorkItem to queue threads for cases when there
aren't any available in the pool. You can set CorSetMaxThreads to
increase the number of available threads in the thread pool. However,
unless you really need all of those threads, I'd leave at the default
25.

The thread pool does a pretty good job of managing all your threads.
Are 25 threads not enough for your app? Using the threadpool limit
will keep your app from draining computer resources and allow other
apps to run more smoothly.

25 threads could be a limit. My app is a service that do jobs requested
by the web application and when many users are using it the more threads
are needed. And also there can be scheduled tasks to be run on specified
time.

Thanks for the response.

Regards,
mircu

 

[Brett Romero]

At that point, having a box that can maintain the demand should
complete the solution.

Brett

 

[Jon Skeet [C# MVP]]

25 threads could be a limit. My app is a service that do jobs requested
by the web application and when many users are using it the more threads
are needed. And also there can be scheduled tasks to be run on specified
time.

Personally I'd avoid using the system thread pool. It's very easy to
deadlock it if your code uses system APIs which might themselves use
the thread pool - which isn't always obvious. I'd recommend using a
custom thread pool. There are plenty of free/open source ones
available, including mine at
http://www.pobox.com/~skeet/csharp/miscutil

 

[Brett Romero]

That's interesting about the thread pool. Is it safe to use the thread
pool when your app isn't doing DLLImport calls? Is that what you mean
by system APIs?

Where did you find the information about System APIs and the thread
pool?

Thanks,
Brett

 

[Jon Skeet [C# MVP]]

That's interesting about the thread pool. Is it safe to use the thread
pool when your app isn't doing DLLImport calls? Is that what you mean
by system APIs?

No - I mean various calls within the normal framework. Even some calls
which don't do anything asynchronously do their work by just executing
asynchronously and waiting for the call to complete. (I believe
WebRequest is an example, but I can't remember for sure at the moment.)

Basically, because anything *can* use the system thread pool, you're
never going to know for sure what *does* use the system thread pool -
so in order to avoid deadlock, I believe it's worth creating your own
thread pool which you know nothing else will use because they won't
know about it. (If MS provided a way of creating *instances* of thread
pools, that would solve the problem. Unfortunately, they just provided
a single thread pool.)

Where did you find the information about System APIs and the thread
pool?

Through the bitter experience of colleagues

 

[Brett Romero]

Funny how this is virtually undocumented given the problems it can
cause. Good info. Thanks.

Brett