multi threading in C#

[Guest]

Hello Everyone,

I just started learning Threading in C#. I read lot of articles and what I
am trying to accomplish here is

that I have an array of string and I want to pass the member of that array
to each thread one by one until the processing is done.
how can I pass the values one by one to myThreadprocess1 and as soon as one
thread is done with the process,
it should grab another item from array and process that item. How can I
accomplish this? Below is my code:



private void processTest()
{

string[] x = { 1234, 6789, 35678, 6789, 023456};
AutoResetEvent MyThread1 = new AutoResetEvent(false);
AutoResetEvent MyThread2 = new AutoResetEvent(false);

ThreadPool.QueueUserWorkItem(new WaitCallback(MyThreadProcess1,
x[0]), MyThread1);
ThreadPool.QueueUserWorkItem(new WaitCallback(MyThreadProcess2,
x[1]), MyThread2);

MyThread1.WaitOne();
MyThread2.WaitOne();


}

static void MyThreadProcess1(Object state)
{


}

 

[Jon Skeet [C# MVP]]

I just started learning Threading in C#. I read lot of articles and what I
am trying to accomplish here is

that I have an array of string and I want to pass the member of that array
to each thread one by one until the processing is done.
how can I pass the values one by one to myThreadprocess1 and as soon as one
thread is done with the process,
it should grab another item from array and process that item. How can I
accomplish this? Below is my code:

It sounds like you want a producer/consumer queue, or something like
it. See the second half of this page:

http://pobox.com/~skeet/csharp/threads/deadlocks.shtml

 

[Peter Duniho]

I just started learning Threading in C#. I read lot of articles and
what I am trying to accomplish here is

that I have an array of string and I want to pass the member of that
array
to each thread one by one until the processing is done.
how can I pass the values one by one to myThreadprocess1 and as soon as
one
thread is done with the process,
it should grab another item from array and process that item. How can I
accomplish this? [...]

In addition to what Jon wrote (you should definitely read his web
article), I have to ask if you can clarify your goal.

I agree that you seem to have a sort-of producer/consumer setup. But it
seems like you only have one producer, and depending on what you mean by
"pass the member of that array to each thread one by one until the
processing is done", maybe even only one consumer.

But, assuming you really mean to have multiple threads working, all of
which pull a new value from your array when they've finished the one
they're working on, then you have a producer/consumer scenario with one
producer (or possibly just an initial set of data) and multiple consumers
(each thread working on the data is a consumer). Using Jon's
producer/consumer code as a starting point, you'd add elements from your
string array to the queue with the Produce() method, and have as many
threads as you want working on them instead of just the one he shows in
his example code (each using Consume() to get a new value to work on).

Keeping in mind, of course, that to get true benefits from multithreading,
you either need to be running on a computer where you have as many CPUs as
you have threads working on the problem, or you have some processing that
is mostly waiting on i/o and want to make sure you always have a thread
ready to accept new i/o when it occurs. Just throwing more threads at
some random problem is not necessarily going to speed things up, and in
many cases can actually slow things down.

Pete

 

[Peter Duniho]

It sounds like you want a producer/consumer queue, or something like
it. See the second half of this page:

http://pobox.com/~skeet/csharp/threads/deadlocks.shtml

By the way, as long as we're on the topic, there's been something I've
been curious about for awhile. I see in your example code, and I've seen
this elsewhere as well, the creation of a new, base object instance for
use as a locking object. I am curious what the point of that is, as
compared to simply acquiring a lock on the object you are trying to
synchronize access to.

For example, instead of using "listLock" for the lock() and Monitor
methods, why not just use the "queue" instance?

The only thing that comes to mind is that in some cases, you may have an
object where only certain parts of it need to be synchronized, but those
parts may still be more than a single object within the larger object. In
that case, having a dedicated object instance for locking allows for
separation of the locking semantics from the actual implementation.

But otherwise? Are there other good reasons for doing it that way, or is
it just a matter of being consistent since you would need do it that way
sometimes?

Thanks,
Pete

 

[Moty Michaely]

By the way, as long as we're on the topic, there's been something I've
been curious about for awhile. I see in your example code, and I've seen
this elsewhere as well, the creation of a new, base object instance for
use as a locking object. I am curious what the point of that is, as
compared to simply acquiring a lock on the object you are trying to
synchronize access to.

For example, instead of using "listLock" for the lock() and Monitor
methods, why not just use the "queue" instance?

The only thing that comes to mind is that in some cases, you may have an
object where only certain parts of it need to be synchronized, but those
parts may still be more than a single object within the larger object. In
that case, having a dedicated object instance for locking allows for
separation of the locking semantics from the actual implementation.

But otherwise? Are there other good reasons for doing it that way, or is
it just a matter of being consistent since you would need do it that way
sometimes?

Thanks,
Pete

Hi Pete,

I can't think of anything besides what you wrote.

I would synchronize the Queue and use the SyncRoot in case I would
like to lock the access to the Queue it self or lock the queue for
general locking purposes. For specific critical section locking (for
example locking the section that adds an item to the queue) I would
use specific objects (static, for global sync).

Moty

 

[Jon Skeet [C# MVP]]

By the way, as long as we're on the topic, there's been something I've
been curious about for awhile. I see in your example code, and I've seen
this elsewhere as well, the creation of a new, base object instance for
use as a locking object. I am curious what the point of that is, as
compared to simply acquiring a lock on the object you are trying to
synchronize access to.

For example, instead of using "listLock" for the lock() and Monitor
methods, why not just use the "queue" instance?

If you lock on a reference that only you have access to, that
guarantees that no-one else can screw things up by locking on it
themselves.

 

[Peter Duniho]

If you lock on a reference that only you have access to, that
guarantees that no-one else can screw things up by locking on it
themselves.

Okay, the way I read that is that in the "listLock" versus "queue"
reference example, "listLock" is the reference only you have access to,
and the implication is that "queue" is a reference that others have access
to.

Uh. Maybe I'm missing something, but...if you lock on a reference that
only you have access to, but the object you're protecting is visible to
others, doesn't that also mean that an alternate way for someone else to
screw things up is by accessing the object *without* locking it?

It seems to me that if you've got an object that needs locking, and it
somehow becomes accessible to outside code not guaranteed to obey the
correct locking semantics, you've got problems one way or the other.
Either the outside code locks it when you don't want it to, or it doesn't
lock it when it ought to.

Isn't the correct solution to actually hide the object that needs locking
from other code? That way the other code can't screw with it one way or
the other.

I feel like something just went over my head, but I'm not sure what it is.

Pete

 

[Jon Skeet [C# MVP]]

Okay, the way I read that is that in the "listLock" versus "queue"
reference example, "listLock" is the reference only you have access to,
and the implication is that "queue" is a reference that others have access
to.

Only the ProducerConsumer class has access to listLock. The general
public (as it were) doesn't have access to the queue reference - but
the queue itself does. If it chose to lock on itself and call
Monitor.Wait(this) for instance, that could screw up the signalling in
the rest of the code. I think it's highly unlikely that it would do
that, but it's a possibility.

(It doesn't help that there are two variables called queue here - I'm
talking about the one inside ProducerConsumer, not the one referring
to ProducerConsumer itself!)

I also like it from an encapsulation point of view - it makes it easy
to replace listLock with a different type of locking type, eg the one
at http://www.yoda.arachsys.com/csharp/miscutil/usage/locking.html

Fundamentally, I don't think that you should be able to lock/wait/
pulse/etc on any old object reference - separating the lock from the
queue makes "my" world closer to my ideal world

(I wonder whether there's some way of encapsulating both parts in one
variable neatly with generics though... I'll have to think about
that.)

Jon

 

[Moty Michaely]

Only the ProducerConsumer class has access to listLock. The general
public (as it were) doesn't have access to the queue reference - but
the queue itself does. If it chose to lock on itself and call
Monitor.Wait(this) for instance, that could screw up the signalling in
the rest of the code. I think it's highly unlikely that it would do
that, but it's a possibility.

(It doesn't help that there are two variables called queue here - I'm
talking about the one inside ProducerConsumer, not the one referring
to ProducerConsumer itself!)

I also like it from an encapsulation point of view - it makes it easy
to replace listLock with a different type of locking type, eg the one
athttp://www.yoda.arachsys.com/csharp/miscutil/usage/locking.html

Fundamentally, I don't think that you should be able to lock/wait/
pulse/etc on any old object reference - separating the lock from the
queue makes "my" world closer to my ideal world

(I wonder whether there's some way of encapsulating both parts in one
variable neatly with generics though... I'll have to think about
that.)

Jon

Hi Jon,

I think it depands on the definition of Ideal World

But your definition sounds rational.

Moty

 

[Peter Duniho]

Only the ProducerConsumer class has access to listLock. The general
public (as it were) doesn't have access to the queue reference - but
the queue itself does. If it chose to lock on itself and call
Monitor.Wait(this) for instance, that could screw up the signalling in
the rest of the code. I think it's highly unlikely that it would do
that, but it's a possibility.

Okay, maybe it's just because I'm up too late, but I still not sure I get
it.

Let's assume the Queue class is defined as not thread-safe. After all, if
it were thread-safe you wouldn't have to do all this work yourself,
right? But then, if the class is defined as not thread-safe, why would it
be doing any locking itself?

And conversely, if it is doing locking itself, wouldn't it be
thread-safe? (That is, at least nominally...not counting bugs in its
implementation of thread-safeness, of course ). And so by implication,
if it's doing locking, why would you be doing locking as well?

Interestingly (at least to me, anyway) at least part of the issue comes
about because of the use of the Monitor class, which apparently has
knowledge of the compiler keyword "lock". While I see the design
simplicity of having the Monitor class, it does introduce some hiding of
implementation details that produce less-than-intuitive results. In
particular, the first time I saw code using Monitor, I had no idea why the
code didn't deadlock, because I didn't understand that calling Wait()
released the lock (all I saw was two different sections of code waiting on
the same object, with no apparent releasing semantics).

But the other odd thing is that provides an avenue for code unrelated to
the higher-level code that did the original lock to wind up unlocking the
lock, unbeknownst to the code that locked the object in the first place.
Of course, I suppose that's the "cost" of having a built-in locking
mechanism, but because of the way "lock" appears in code, it's a little
confusing that you can be within the block of code contained by the "lock"
statement, and still not be locked.

Which is, I guess, your point about using a different object. That is,
while from a design perspective it seems *extremely* unlikely that you'd
have a class that required you to do locking, but which still wound up
doing locking (or unlocking, in this case) itself that screwed you up, it
is technically possible and thus the use of an independent object to do
the locking instead.

So, after all that...I guess my understanding is that the use of the
object is about what's technically possible, rather than what is likely.
Belt and suspenders, and all that. Is that right?

Pete

 

[Christof Nordiek]

Uh. Maybe I'm missing something, but...if you lock on a reference that
only you have access to, but the object you're protecting is visible to
others, doesn't that also mean that an alternate way for someone else to
screw things up is by accessing the object *without* locking it?

This would be possible in both cases. Making the lock object publicly
accessible doesn't force callers to lock.
The trick is, to implement the class in such a way, that it can be used
safely without locking in any multithreaded implementation.

Christof

 

[Jon Skeet [C# MVP]]

Okay, maybe it's just because I'm up too late, but I still not sure I get
it.

Let's assume the Queue class is defined as not thread-safe. After all, if
it were thread-safe you wouldn't have to do all this work yourself,
right? But then, if the class is defined as not thread-safe, why would it
be doing any locking itself?

It may be doing locking in a different way, or for some different
purpose. It may have some internal data structure that it needs a lock
for, but without making the whole thing thread-safe.

Also, it might *become* thread-safe in a later version by introducing
locking. I'm probably being overcautious in this particular case, but
it's because I'm applying a general principle. I don't like having to
do too much reasoning about threading - the simpler rules, the better,
IMO.

Interestingly (at least to me, anyway) at least part of the issue comes
about because of the use of the Monitor class, which apparently has
knowledge of the compiler keyword "lock".

The reverse is true - "lock" is like "using", but calls Monitor.Enter/
Monitor.Exit.

While I see the design
simplicity of having the Monitor class, it does introduce some hiding of
implementation details that produce less-than-intuitive results. In
particular, the first time I saw code using Monitor, I had no idea why the
code didn't deadlock, because I didn't understand that calling Wait()
released the lock (all I saw was two different sections of code waiting on
the same object, with no apparent releasing semantics).

Right - but I'm not sure I'd call that an implementation detail. It's
part of the public design of monitors. It's true that you need to
understand monitors in order to use them properly, but I don't think
that's too bad.

But the other odd thing is that provides an avenue for code unrelated to
the higher-level code that did the original lock to wind up unlocking the
lock, unbeknownst to the code that locked the object in the first place.

True. Which is one reason for having a separate lock in my case - if
Queue did Monitor.Wait(this) inside it, all hell could break loose in
my code. I don't for a minute think it actually does, but I like to be
consistent.

Of course, I suppose that's the "cost" of having a built-in locking
mechanism, but because of the way "lock" appears in code, it's a little
confusing that you can be within the block of code contained by the "lock"
statement, and still not be locked.

Indeed - I wish lock didn't exist, and that something *like* my
locking system (but no doubt much better) had been introduced to start
with. It makes the whole thing more explicit.

Which is, I guess, your point about using a different object. That is,
while from a design perspective it seems *extremely* unlikely that you'd
have a class that required you to do locking, but which still wound up
doing locking (or unlocking, in this case) itself that screwed you up, it
is technically possible and thus the use of an independent object to do
the locking instead.

Yup. It's more worrying when you lock on "this" yourself - in that
anyone with an instance of your class can lock on the same reference -
but this is just a smaller example of the same worry.

So, after all that...I guess my understanding is that the use of the
object is about what's technically possible, rather than what is likely.
Belt and suspenders, and all that. Is that right?

Belt, suspenders, and applying what I think of as design principles
even where they may not be utterly necessary

Jon

 

[Chris Mullins [MVP]]

I see in your example code, and I've seen this elsewhere as well, the
creation of a new, base object instance for use as a locking object. I
am curious what the point of that is, as compared to simply acquiring a
lock on the object you are trying to synchronize access to.

This is something I do as well, and force the developers who are stuck
working with me to follow.

The idea being, if i'm going to do locking, I want it to be explicit. This
means:

private object _syncroot = new object();
public object SyncRoot { get { return _syncroot;} }

.... and all locking is then done via: lock(this.SyncRoot).

This has a few advanced over the mechanism you're talking about, which is:
lock (_myList){...}

The advantages (to me) are:
- Debugging is much easier as I can easily put a breakpoint on the property
accessor and see who is acquiring the lock. On more than one occasion, I've
added a "Debug.WriteLine(StackTrace)" to that method, so I could see exactly
what was going on and in what order.

- If I want to change the locking mechanism out from a Monitor to a
ReaderWriterLock or a ReaderWriterLockSlim, it's easier. I can change the
return type on the property accessor, and let the compiler tell me all the
places that I need to fix.

- It's more explicit, in the "each variable has exactly 1 purpose" sense. I
actually don't like that you can lock on any object, as it seems to lead to
developers doing some wild stuff. I see "lock(this)" quite often, which is
just a recipe for disaster.

- It's easier to expose a lock to other classes if I'm taking this approach.
I really don't want to have other classess doing locks on my collection, as
it makes things very unpredictable (see: Debugging is much easier).

I usually throw the "DebuggerHidden" attribute on the accessor, as I don't
like stepping into it by mistake all the time.

For anyone interested, I wrote up (what I think) is a nice pattern for
Enumerating Collections in a threaded world. I also delved a bit into the
Anti-Patterns that I've stumbled across the hard way:
http://www.coversant.com/Coversant/Blogs/tabid/88/EntryID/40/Default.aspx

 

[Chris Mullins [MVP]]

Fundamentally, I don't think that you should be able to lock/wait/
pulse/etc on any old object reference - separating the lock from the
queue makes "my" world closer to my ideal world

I just wrote that (practically word for word) in another post. I've been
reading your posts for too long, I guess...

I feel like Professor Chaos (aka: Butters) from South Park running around
trying to come up with an original diaobicol deed, only to have General
Disarray keep saying, "Simpson's did it! Simpson's did it!"

 

[Jon Skeet [C# MVP]]

I just wrote that (practically word for word) in another post. I've been
reading your posts for too long, I guess...

I feel like Professor Chaos (aka: Butters) from South Park running around
trying to come up with an original diaobicol deed, only to have General
Disarray keep saying, "Simpson's did it! Simpson's did it!"

I'd feel all special about that if it weren't for the fact that the
idea isn't exactly new to me in the first place. You've probably read
me waffling on about it more than others just through circumstance, but
it's a fairly well-worn road these days

(I think it was fairly well-worn in the Java community back in 2000 or
so when .NET still hadn't been released - I'm slightly surprised they
didn't look more closely. Maybe there are advantages we're both missing
though.)