Puzzeling .NET behavior with signaling (ThreadPool,RegisterWaitForSingelObject and AutoResetEvent)

[solandre]

I have encountered some very puzzeling behavior of .NET which I'm
seeking an explanation for:

This is just a test program to check out how
ThreadPool.RegisterWaitForSingelObject works.

The following code ....

class Program
{
static int NumRegisters=2;
static RegisteredWaitHandle[] rwh = new
RegisteredWaitHandle[Program.NumRegisters];

static void Main(string[] args)
{
AutoResetEvent wh = new AutoResetEvent(false);
for (int i = 0; i < Program.NumRegisters; i++)
{
Program.rwh =
ThreadPool.RegisterWaitForSingleObject(wh, Program.Do, i, 2000, true);
}
wh.Set();
wh.Set();
Console.WriteLine("Press any key to exit.");
Console.ReadKey();
}

static void Do(object o, bool TimedOut)
{
Console.Write("Register {0}: ",(int)o);
if (TimedOut)
{
Console.WriteLine("Timeout while waiting for
signal.");
}
else
{
Console.WriteLine("Procedure started. Doing something
and exiting.");
//Thread.Sleep(1000);
}
}
}


yields this output:


Register 0: Procedure started. Doing something and exiting.
Press any key to exit.
Register 1: Timeout while waiting for signal.


Very intersting, as there is sent a signal two times, but just one
registered procedure is started.
If I insert a Thread.Sleep(0); between the two signals .... (the
following code is the same as above, just with one additional line)


class Program
{
static int NumRegisters=2;
static RegisteredWaitHandle[] rwh = new
RegisteredWaitHandle[Program.NumRegisters];

static void Main(string[] args)
{
AutoResetEvent wh = new AutoResetEvent(false);
for (int i = 0; i < Program.NumRegisters; i++)
{
Program.rwh =
ThreadPool.RegisterWaitForSingleObject(wh, Program.Do, i, 2000, true);
}
wh.Set();
Thread.Sleep(0);
wh.Set();
Console.WriteLine("Press any key to exit.");
Console.ReadKey();
}

static void Do(object o, bool TimedOut)
{
Console.Write("Register {0}: ",(int)o);
if (TimedOut)
{
Console.WriteLine("Timeout while waiting for
signal.");
}
else
{
Console.WriteLine("Procedure started. Doing something
and exiting.");
//Thread.Sleep(1000);
}
}
}


it works as expected:


Register 0: Procedure started. Doing something and exiting.
Register 1: Procedure started. Doing something and exiting.
Press any key to exit.


Anyone who can explain why?

 

[Peter Duniho]

[...]
Very intersting, as there is sent a signal two times, but just one
registered procedure is started.

This is your mistake. It's not true that "there is sent a signal two
times". That's not how the WaitEventHandle works (AutoResetEvent in your
case). The handle has two states: "set", and "not set". If the handle is
set multiple times before any other thread gets a chance to run, that's
exactly the same as setting it once.

You're not sending a signal, you're simply changing the state of the
event. That change of state can only be observed by one thread at a time
(whichever thread is first in the wait queue for the event), and no
subsequent change of state will occur until some thread has observed the
first change of state and reset the event to "not set", allowing another
change of state to occur.

In other words, if you want to use the thread pool this way, you need to
include logic to ensure that you don't try to set the event again until
you're sure that some thread that was waiting on the event has in fact had
a chance to be released from the event, causing the event to be set back
to "not set" and a new thread to get to the front of the wait queue and be
waiting on the event.

When you call Sleep(0), you tell Windows that if there are any threads
waiting to run at the same or higher priority as the current thread, they
should be allowed to run. So by adding the call to Sleep(0), the thread
setting the event yields, the first waiting thread gets to run, resetting
the event handle and allowing the second thread to become the new head of
the wait queue. Eventually, your main thread gets to run again, but by
that time all of the resetting/queueing has happened and setting the wait
handle again can actually be observed by the second waiting thread.

I'm a bit leery of that particular solution, because it makes an
assumption that the thread at the head of the wait queue for that event
_will_ be run when you call Sleep(0). This is in fact a pretty reasonable
assumption, as long as you keep the thread priorities equal (which they
are if you don't modify them), and as long as the current Windows thread
schedule implementation doesn't change (and it's highly unlikely that it
would change in a way that would break this).

But it's still an assumption, and I prefer not to have assumptions like
that. IMHO, it would be better to design the code so that when you _know_
for sure that the first waiting thread has had a chance to run, _then_ you
set the event again, and only then. The only way to know for sure is to
handle that inside the code executing in that thread. You could use a
second event, set from the waiting thread when it gets to run, and waited
on by the main thread. Alternatively, you could just have the waiting
thread set the original event again once it gets to run, releasing the
next thread in line (presumably in real code, you only actually set the
event when there's some specific condition that should release a queued
thread pool delegate).

IMHO, the second approach is better, as it avoids having the original main
thread having to execute when not necessary. But either should work fine.

Finally, this all assumes that using this particular mechanism in the
thread pool is in fact the right way to do things. It's hard to say for
sure that it is without knowing exactly what these threads would be doing
in a real-world scenario. So I'll take as granted that technique, which
is potentially a little more complex than other synchronization
techniques, is appropriate for your situation.

Pete

 

[solandre]

hey pete,

thank you very much for your fast and extensive reply. It's
illustrative and gives me insight beyond the developers foundations
curriculum (and i assume for your quicknes you read my post right
after i wrote it and started typing).

I knew the AutoResetEvent let pass one thread at a time, but I didn't
realize, that the state can just be seen by the frontmost thread until
it is through (had a chance to run). And thanks for pointing out the
assumptions underlying the Sleep(0). Further puzzeling results
regarding that are foreclosed.

 

[Jeroen Mostert]

I have encountered some very puzzeling behavior of .NET which I'm
seeking an explanation for:

This is just a test program to check out how
ThreadPool.RegisterWaitForSingelObject works.

The following code ....

class Program
{
static int NumRegisters=2;
static RegisteredWaitHandle[] rwh = new
RegisteredWaitHandle[Program.NumRegisters];

static void Main(string[] args)
{
AutoResetEvent wh = new AutoResetEvent(false);
for (int i = 0; i < Program.NumRegisters; i++)
{
Program.rwh =
ThreadPool.RegisterWaitForSingleObject(wh, Program.Do, i, 2000, true);
}

In addition to everything Pete's said which probably makes the problem moot
anyhow, you cannot use the same handle in multiple class to
..RegisterWaitForSingleObject(). The MSDN clearly documents this:

The wait thread uses the Win32 WaitForMultipleObjects function to monitor
registered wait operations. Therefore, if you must use the same native
operating system handle in multiple calls to RegisterWaitForSingleObject,
you must duplicate the handle using the Win32 DuplicateHandle function.

 

[Peter Duniho]

[...]
The wait thread uses the Win32 WaitForMultipleObjects function to
monitor registered wait operations. Therefore, if you must use the same
native operating system handle in multiple calls to
RegisterWaitForSingleObject, you must duplicate the handle using the
Win32 DuplicateHandle function.

It does say that. And in fact, the docs for WaitForMultipleObjects and
WaitForMultipleObjectsEx both stipulate that you cannot use the same event
handle more than once in the passed-in array to those functions.

And yet, it seems to work. I wrote a little C++ test program to test the
native functions directly, and it didn't complain when I passed in the
same handle twice in the array of event handles. The thread waited fine,
and was released when I set the handle.

I thought the .NET thread pool just used the native thread pool anyway, so
I'm not sure why it's not using the native RegisterWaitForSingleObject()
anyway. But assuming the docs are correct and it's not, I'm not sure why
it's actually an error to reuse the same event handle. I agree the docs
say it is, both in the .NET docs for
ThreadPool.RegisterWaitForSingleObject() as well as for the Win32 docs for
WaitForMultipleObjectsEx and WaitForMultipleObjects. But I'd love to know
_why_ the docs say that, and why it seems to work even though they say
it's an error to do so.

I realize this is pretty much off-topic here now, but if anyone can in
just one more reply explain why you're not supposed to use the same event
handle twice in a single call to WaitForMultipleObjects, and especially
can answer why even though it's not allowed, no error happens when you do,
I think it'd be worth posting here for our edification.

Pete

 

[Jeroen Mostert]

[...]
The wait thread uses the Win32 WaitForMultipleObjects function to
monitor registered wait operations. Therefore, if you must use the
same native operating system handle in multiple calls to
RegisterWaitForSingleObject, you must duplicate the handle using the
Win32 DuplicateHandle function.

It does say that. And in fact, the docs for WaitForMultipleObjects and
WaitForMultipleObjectsEx both stipulate that you cannot use the same
event handle more than once in the passed-in array to those functions.

And yet, it seems to work. I wrote a little C++ test program to test
the native functions directly, and it didn't complain when I passed in
the same handle twice in the array of event handles. The thread waited
fine, and was released when I set the handle.

Sure, sure... But the MSDN says "do not do that", and if they actually go so
far as to hoist this warning over to the .NET documentation, we should
probably respect it. Have you tested all Windows flavors? All versions? All
scenarios, particularly multiple threads? All types of waitable objects?

This comment may be broader then necessary, because someone didn't want to
commit themselves to supporting all scenarios (some possibly very subtle),
or because the problems with this have been fixed but it's never been made
official.

I thought the .NET thread pool just used the native thread pool anyway,
so I'm not sure why it's not using the native
RegisterWaitForSingleObject() anyway.

Actually, it *is* using that. The native RegisterWaitForSingleObject()
passes the request on to the thread pool's wait threads, and those threads
use WaitForMultipleObjectsEx(). Disassemble the function if you're
interested, it's actually quite interesting (the wait threads are woken up
with an APC whenever the list of objects to wait for changes). I'm not sure
why the .NET documentation feels it has to go into such detail when the
documentation for the Win32 RegisterWaitForSingleObject() does not, though.

But assuming the docs are correct and it's not, I'm not sure why it's
actually an error to reuse the same event handle. I agree the docs say it
is, both in the .NET docs for ThreadPool.RegisterWaitForSingleObject() as
well as for the Win32 docs for WaitForMultipleObjectsEx and
WaitForMultipleObjects. But I'd love to know _why_ the docs say that, and
why it seems to work even though they say it's an error to do so.

I agree, it would be interesting to know this. I'm following this up to
comp.os.ms-windows.programmer.win32. May be more of a kernel question,
actually, but let's not get premature.

 

[Willy Denoyette [MVP]]

[...]
The wait thread uses the Win32 WaitForMultipleObjects function to
monitor registered wait operations. Therefore, if you must use the same
native operating system handle in multiple calls to
RegisterWaitForSingleObject, you must duplicate the handle using the
Win32 DuplicateHandle function.

It does say that. And in fact, the docs for WaitForMultipleObjects and
WaitForMultipleObjectsEx both stipulate that you cannot use the same event
handle more than once in the passed-in array to those functions.

And yet, it seems to work. I wrote a little C++ test program to test the
native functions directly, and it didn't complain when I passed in the
same handle twice in the array of event handles. The thread waited fine,
and was released when I set the handle.

I thought the .NET thread pool just used the native thread pool anyway, so
I'm not sure why it's not using the native RegisterWaitForSingleObject()
anyway. But assuming the docs are correct and it's not, I'm not sure why
it's actually an error to reuse the same event handle. I agree the docs
say it is, both in the .NET docs for
ThreadPool.RegisterWaitForSingleObject() as well as for the Win32 docs for
WaitForMultipleObjectsEx and WaitForMultipleObjects. But I'd love to know
_why_ the docs say that, and why it seems to work even though they say
it's an error to do so.

I realize this is pretty much off-topic here now, but if anyone can in
just one more reply explain why you're not supposed to use the same event
handle twice in a single call to WaitForMultipleObjects, and especially
can answer why even though it's not allowed, no error happens when you do,
I think it'd be worth posting here for our edification.

Pete

The CLR does not use the OS TP, he uses his own implementation that was
based on W2K's TP.
That means that he doesn't call the Win32
Kernel32!RegisterWaitForSingleObject instead he calls his own implementation
(mscorwks!ThreadPoolMgr::RegisterWaitForSingleObject). Granted the waits are
done by calling into Win32's WaitForMultipleObjects, so you need to consider
the "same handle" remark with care. Registering the same handle multiple
times, requires the handle to stay valid for as long he remains registered,
that means that you cannot consider ownership, so you cannot close/destroy
the handle without the risk to invalidate another "waiter". The only way to
prevent this is by duplicating the handle (and take ownership).

Willy.

 

[Jeroen Mostert]

Willy Denoyette [MVP] wrote

The CLR does not use the OS TP, he uses his own implementation that was
based on W2K's TP.
That means that he doesn't call the Win32
Kernel32!RegisterWaitForSingleObject instead he calls his own
implementation (mscorwks!ThreadPoolMgr::RegisterWaitForSingleObject).

Oops. I sort of assumed it *did* use the Win32 implementation when I saw the
"internalcall", but of course that doesn't say anything. It sounds logical
when you consider .NET has to run on Win95/98 as well, and those don't
*have* native threadpools. It's easier to roll your own than to
conditionally defer to the OS, I suppose.

Granted the waits are done by calling into Win32's
WaitForMultipleObjects, so you need to consider the "same handle" remark
with care. Registering the same handle multiple times, requires the
handle to stay valid for as long he remains registered, that means that
you cannot consider ownership, so you cannot close/destroy the handle
without the risk to invalidate another "waiter". The only way to prevent
this is by duplicating the handle (and take ownership).

But the help already explicitly mentions that the function's behavior is
undefined if you close a handle while it's being waited on. It seems that if
this is the only reason, waiting on the same handle multiple times is fine
as long as you don't close it, but this is not what the help says.

 

[Willy Denoyette [MVP]]

Willy Denoyette [MVP] wrote


Oops. I sort of assumed it *did* use the Win32 implementation when I saw
the "internalcall", but of course that doesn't say anything. It sounds
logical when you consider .NET has to run on Win95/98 as well, and those
don't *have* native threadpools. It's easier to roll your own than to
conditionally defer to the OS, I suppose.

Well, the CLR was written back when W2K was in beta (XP's develpment was not
even started), and W2K was the first OS supporting a TP, down-level OS'ses
did not have a TP implementation.
So, to support W98 and NT4, the CLR team had to "virtualize" a TP (just
like some other services :-( ) and they based this one on W2K's.
Unfortunately very little has changed since then, the OS TP evolved
considerably, the latest (Vista and WS2008) TP implementations are superior
to the CLR's virtualized TP, I hope MS is now busy to bring the CLR back on
par with the latest OS'ses and I hope they drop downlevel OS (tahis is
anything lower than XP) support in "CLR next".

Granted the waits are done by calling into Win32's
But the help already explicitly mentions that the function's behavior is
undefined if you close a handle while it's being waited on. It seems that
if this is the only reason, waiting on the same handle multiple times is
fine as long as you don't close it, but this is not what the help says.

Well, this is one of the reason's but there are others, all depends on the
type of handle used. Some handles cannot be "waited" on, some are
auto-closing, others won't be used from "pure/safe" managed code, for
instance in "unmanaged", one could register a native thread's handle
(returned by _beginthread()), however such handles are auto-closed when the
thread terminates (_endthread ).

Willy.

 

[Peter Duniho]

[...]

But the help already explicitly mentions that the function's behavior
is undefined if you close a handle while it's being waited on. It seems
that if this is the only reason, waiting on the same handle multiple
times is fine as long as you don't close it, but this is not what the
help says.

Well, this is one of the reason's but there are others, all depends on
the type of handle used. Some handles cannot be "waited" on, some are
auto-closing, others won't be used from "pure/safe" managed code, for
instance in "unmanaged", one could register a native thread's handle
(returned by _beginthread()), however such handles are auto-closed when
the thread terminates (_endthread ).

So the advice may not in fact pertain specifically to an event handle, but
rather to handles that more generally could be passed to the function.

For what it's worth, I did some Googling earlier, and came up with a
couple of oddly incongruous references. In one, an older version of the
SDK documentation for WaitForMultipleObjects(), not only was there no
mention of this restriction against using the same handle twice, it
specifically said that on Win95 you could NOT pass in a handle that had
been created with DuplicateHandle() (the proposed solution in the current
docs for dealing with wanting to use the same object twice).

In another, in the context of comments for an author's book, he wrote that
if you do pass the same handle multiply to the function, it will return an
"invalid parameter" error. I can say with certainty that this isn't true
in WinXP SP2, when using an event handle. But it may be true in some
other situation (though, it would be non-trivial to do that check without
incurring a serious performance problem and so there's every possibility
that the OS simply will not check this for you, and instead just behave
unreliably when you do the wrong thing).

Finally, as I sit here pondering "why wouldn't you be able to pass the
same handle twice", I start thinking about other kinds of handles. In
particular, a kind where releasing a thread based on that handle more than
once could be problematic. For simple resettable handles, like an event,
I don't see a problem. For handles that are never going to cause a wait
again, like a thread handle, I don't see a problem.

But what about a semaphore?

WaitForMultipleObjects says that it will sit and wait until _all_ of the
handles have satisfied their wait condition simultaneously, and then when
that moment occurs, it will go through all of the handles and do whatever
"unsignaling" is appropriate for each handle. For an auto-reset event
handle, being unset multiple times isn't likely to be a problem, but for a
semaphore I can see how it might cause problems to get that semaphore more
than once.

But frankly, this still doesn't explain the WaitForMultipleObjects
admonishment to me. After all, it's not WaitForMultipleObjects job to
prevent bugs...it should just do whatever the code tells it to do, even if
it's not correct. And in fact, it could be possible to write code that
correctly deals with the multiple acquisition of the semaphore by a thread
in one shot. In other words, even in the case of the semaphore I don't
see what would break in WaitForMultipleObjects specifically, even if it
would in fact usually be a bad idea to pass the same semaphore more than
once to it.

None of which, of course, proves that the docs are wrong. It just means I
still haven't seen the actual explanation for why you can't use the same
handle twice in that call.

Oh well. It's not like there aren't other mysteries in the OS for me
either. I completely agree that in this matter, it's much more important
to just do what the docs say, than to worry about why they say it.

Pete

 

[Peter Duniho]

[...] Registering the same handle multiple times, requires the handle to
stay valid for as long he remains registered, that means that you cannot
consider ownership, so you cannot close/destroy the handle without the
risk to invalidate another "waiter". The only way to prevent this is by
duplicating the handle (and take ownership).

Well, that's a nice explanation for why you need to duplicate a handle you
don't own before passing it to the function.

But for a handle you already own, it doesn't tell me why you can't include
that handle more than once in the array passed to WaitForMultipleObjects.

Pete

 

[Willy Denoyette [MVP]]

Willy Denoyette [MVP] wrote


Oops. I sort of assumed it *did* use the Win32 implementation when I saw
the "internalcall", but of course that doesn't say anything. It sounds
logical when you consider .NET has to run on Win95/98 as well, and those
don't *have* native threadpools. It's easier to roll your own than to
conditionally defer to the OS, I suppose.

But the help already explicitly mentions that the function's behavior is
undefined if you close a handle while it's being waited on. It seems that
if this is the only reason, waiting on the same handle multiple times is
fine as long as you don't close it, but this is not what the help says.

Yes, but the problem is that some handles may get "auto-closed", but as I
said before this is less of a problem in "unmanaged" code.
Note also that ThreadPool.RegisterWaitForSingleObject description in MSDN is
rather confusing, the "Important Note" makes you think that :
1) the underlying API is kernel32!RegisterWaitForSingleObject, which is not
true.
2) the callbacks are handled by "Worker" threads from the (CLR)TP, which is
not the case, the are handled by IOC threads from the (CLR)TP.

Willy.