Approach to Implement a Thread Pump?

[DC]

Hi,

I need the functionality to take a workitem from a stack, execute it in
a different thread, get the next item, execute it asynchronously too,
and so on. But: there should never be more than 5 threads processing
workitems. As soon as the number of threads gets below five, more
workitems can be queued. This goes on until the stack of workitems is
fully processed. Not sure if that is what one would call a "Thread
Pump".

I am thinking about doing this with a .BeginInvoke() approach and a
static thread counter, but I wanted to check if somebody knows a common
pattern for my task? Also, with my approach I would do a sleep in the
main thread and periodically check if the static counter fell below 5,
but that seems clumsy to me. I don't know how to design a sync object
that can trigger program continuation whenver one of the threads
signaled that its execution finished.

On a sidenote: I am about to (finally) convert this project to
framework 2.0. Will that give me more threading options? I tested the
program (which works for years with framework 1.1) with framework 2.0
btw, and it produced a ton of "out of memory" errors (on the same
machine where it worked fine with 1.1).

Thanks for any hint in advance,
Regards
DC

 

[Larry Lard]

Hi,

I need the functionality to take a workitem from a stack, execute it in
a different thread, get the next item, execute it asynchronously too,
and so on. But: there should never be more than 5 threads processing
workitems. As soon as the number of threads gets below five, more
workitems can be queued. This goes on until the stack of workitems is
fully processed. Not sure if that is what one would call a "Thread
Pump".

No, not really. I'd call it a Producer-Consumer Queue (with 5 Consumers
in this case). And I'd use Jon Skeet's implementation:

I am thinking about doing this with a .BeginInvoke() approach and a
static thread counter, but I wanted to check if somebody knows a common
pattern for my task? Also, with my approach I would do a sleep in the
main thread and periodically check if the static counter fell below 5,
but that seems clumsy to me. I don't know how to design a sync object
that can trigger program continuation whenver one of the threads
signaled that its execution finished.

On a sidenote: I am about to (finally) convert this project to
framework 2.0. Will that give me more threading options? I tested the
program (which works for years with framework 1.1) with framework 2.0
btw, and it produced a ton of "out of memory" errors (on the same
machine where it worked fine with 1.1).

Don't think there are any feature changes to multithreading in 2.0 - I
believe it's stricter about cross-thread no-no's in WinForms, but that's
about it.

 

[Ignacio Machin \( .NET/ C# MVP \)]

Hi,

Adding to the response of Larry,

| I need the functionality to take a workitem from a stack,

Are you sure you want to use a Stack and not a Queue?
Also make sure that the Stack (or Queue) is sync'ed . Both classes support
this already.

| On a sidenote: I am about to (finally) convert this project to
| framework 2.0. Will that give me more threading options? I tested the
| program (which works for years with framework 1.1) with framework 2.0
| btw, and it produced a ton of "out of memory" errors (on the same
| machine where it worked fine with 1.1).

I cannot think of anything different, not in such a way that would provoke a
"out of memory", are you using recursion?

 

[DC]

Thank you, Larry. A good read!

No, not really. I'd call it a Producer-Consumer Queue (with 5 Consumers
in this case). And I'd use Jon Skeet's implementation:



Don't think there are any feature changes to multithreading in 2.0 - I
believe it's stricter about cross-thread no-no's in WinForms, but that's
about it.


--
Larry Lard
(e-mail address removed)
The address is real, but unread - please reply to the group
For VB and C# questions - tell us which version

 

[DC]

Are you sure you want to use a Stack and not a Queue?

Yes, a queueue, "stack" was just easier to type.

I am not using recursion. My main memory consumer is a huge hashtable.
I think this object consumes more memory than it did in framework 1.1
(but in lack of an easy sizeof() operator I am unable to say how many
bytes the hashtable really consumes).

 

[Ignacio Machin \( .NET/ C# MVP \)]

Hi,


| On a sidenote: I am about to (finally) convert this project to
| framework 2.0. Will that give me more threading options?

Frankly I do not see what can have 2.0 of new regarding threading that may
be useful for you., if the process was working in 1.1 it should work the
same in 2.0, but of course only a test could prove this.