Which kind of multithreading?

[ce_clown]

Hi NG,

I'm having a little problem that I cannot solve myself due to missing
experience.

Here's the scenario:
a service starts a number of scheduled tasks that are started as a new
thread, each.

One of these tasks has to prepare a number of data records from
different database tables and transfer these into other tables of a
history database. For maximum effectivity the several actions shall be
processed using multithreading. When all steps are finished the task
reports the successful completion to the service.

Using the Thread Pool all steps could be added to the queue using
QueueUserWorkItem and would be processed then. Here, the question is,
how I can make sure that all steps have been finished so that I know
when the task has successfully finished? Apart from that the Thread
Pool examples in the MSDN suggest to use a sleep in order to prevent
the main thread from finishing before the Thread Pool threads even
start working (a join seems to be possible only for individually
started threads) - or are these examples only so much simplified that
methods for waiting for the tasks to finish were just omitted? As a
result I'm just not sure if can I use the Thread Pool in my case.

The alternative - individually started "subthreads" - represents a
less effective (and probably needing more coding efforts) solution
according to the Thread Pool description, but probably it does not
have the problems described above. What I do not know here is how I
can manage hundreds of actions (how many threads? How do I know when
one thread has finished and can start the next action?)

Maybe somebody can give me a hint how I can best solve my problem. By
the way: the app is developped with VS2003/.NET 1.1.
Thanks.

Kind regards
Roland

 

[Marc Gravell]

Well, I wouldn't do *hundreds* of (simultaneous) actions on the pool -
I would use a queue-server with a handful of worker threads. I believe
Jon has such a beast in his "Misc Utils" (http://www.pobox.com/~skeet/
csharp/miscutil - look at CustomThreadPool).

In terms of knowing when things are all done, well; with the queue
approach you can check the items remaining in the queue (migght need
to allow for the last ones to finish... can't remember if Jon's
implementation exposes this).

Alternatively, just use a counter and Monitor.Pulse / Monitor.Wait,
i.e.

private readonly object SyncLock = new object();
private int remaining;

private void DoWork(int workers) {
lock(SyncLock) {
// TODO: start workers
remaining = workers; // or perhaps +=
while(remaining>0) {
Monitor.Wait(SyncLock);
}
// presto: all have finished
}
}
// call this at the end of each task
private void MarkOneTaskAsCompleted() {
lock(SyncLock) {
remaining--;
if(remaining<=0) {
Monitor.Pulse(SyncLock); // or PulseAll
}
}
}

 

[ce_clown]

Hi Marc,

thank you for your fast answer.

Well, I did not intend to start a thread for each of the "hundreds" of
actions at once. This is why I asked "how many threads?" for the
"individually started threads" solution.
Currently I cannot have a look at Jon's CustomThreadPool but I will as
soon as I can.

As I'm not really familiar with locking and multithreading I will try
to understand your code snippets - as far as I do now it looks
promising!
If I did not succeed I'd probably come back with further questions ...

Regards,
Roland

 

[Marc Gravell]

How many threads? Depends on the situation. If your work is CPU-bound,
then not much point going above the number of cores
(Environment.ProcessorCount or some-such), and even that is if you are
on a dedicated server (i.e. nobody will mind if you steal all the
runtime). If you are IO-bound (disk, network, etc) then you can up it
a bit, but I suspect you will need to experiment a bit to find the
sweet spot.

Marc

 

[Jon Skeet [C# MVP]]

As I'm not really familiar with locking and multithreading I will try
to understand your code snippets - as far as I do now it looks
promising!
If I did not succeed I'd probably come back with further questions ...

If you're not familiar with threading to start with, that should be the
first thing to fix, before you consider thread pools.

See http://pobox.com/~skeet/csharp/threads for more information.

 

[ce_clown]

Hi Jon,

I've been reading your article about threads before and it helped me
to understand the theory and which pitfalls there are (thanks for all
the knowledge you share with other people!) - but when it gets to
coding I still feel rather unconfident with this topic. So when I read
that using the ThreadPool class could save me from doing the most
tricky parts myself, I was interested in using it. But maybe I'll have
to go into the details to better understand how to do these things.

Regards
Roland

 

[Jon Skeet [C# MVP]]

I've been reading your article about threads before and it helped me
to understand the theory and which pitfalls there are (thanks for all
the knowledge you share with other people!) - but when it gets to
coding I still feel rather unconfident with this topic.

In some ways that's good. There's nothing worse than trying to unpick
the code of someone who thinks that putting lock(this) in every method
is the answer to all threading issues!

So when I read
that using the ThreadPool class could save me from doing the most
tricky parts myself, I was interested in using it. But maybe I'll have
to go into the details to better understand how to do these things.

You basically need to work out what your shared data is - what needs to
be seen by threads. Protect that with locks and that's a lot of the
work done. If you need synchronization and coordination, that's when
things get trickier - but we can help you when you get to that stage.