BeginInvoke failing when executed in a form that was created in a worker thread...

[Ian Mac]

I previously posted this in the VB.General forum, but thought that
this one might be a better place for it...sorry about the double post.

I have created a simple form with code that looks like this:

'******************************************************************************************

Public Class FEventRaiser
Inherits System.Windows.Forms.Form

Event MyEvent(ByVal eventName as string)

Private Delegate Sub EventRaiserDelegate(ByVal eventName As String)

Public Sub New()

...other code removed for clarity.

'Force the creation of the form's handle.
While Not IsHandleCreated
Dim temp As IntPtr = Handle
End While

End Sub

Public Sub RaiseAnEvent(ByVal eventName As String)

If Me.InvokeRequired Then
'Call made on thread different from the thread this form was
created on...BeginInvoke.
Me.BeginInvoke(New EventRaiserDelegate(AddressOf
RaiseAnEvent), New
Object() {eventName})
Else
'Call made on same thread as this form was created on...raise
the
event.
RaiseEvent MyEvent(eventName)
End If

End Sub
End Class

'******************************************************************************************

If I create an instance of this form from within another form, and
then make
calls to 'MyRaiseEvent' from within a threaded timer, it works as I
would
expect:
Public Class Form1
Inherits System.Windows.Forms.Form
....
Private WithEvents mEventRaiser as FEventRaiser
Private WithEvents mTimer as System.Timers.Timer
...
mEventRaiser = New FEventRaiser
mTimer = New System.Timers.Timer(1000)
mTimer.Enabled = True
....
Private Sub mEventRaiser_MyEvent(ByVal eventName As String) Handles
mEventRaiser.MyEvent
'This event is handled on the same thread that this form's message
pump is
running on.
Debug.WriteLine("Event processed on thread: " &
AppDomain.GetCurrentThreadId)
End Sub

Private Sub mTimer_Elapsed(ByVal sender As Object, ByVal e As
System.Timers.ElapsedEventArgs) Handles mTimer.Elapsed

mEventRaiser.RaiseAnEvent("This Works")

End Sub

End Class

Even though the timer event handler is firing on different thread IDs
than
the main thread, the call to the mEventRaiser.RaiseAnEvent ends up
performing the necessary BeginInvoke, which results in mEventRaiser
raising
an event back on the main thread...this is the behaviour I would
expect (and
want).

If I then modify this code somewhat so that the mEventRaiser is
actually
created once in the timer event handler, then it all stops working
(please
ignore the hack method I used for instantiating the CEventRaiser
object...the real way it's done is in a worker thread created at the
same
time the threaded timer is created). Here's the new timer event
handler:

Private Sub mTimer_Elapsed(ByVal sender As Object, ByVal e As
System.Timers.ElapsedEventArgs) Handles mTimer.Elapsed

If mEventRaiser Is Nothing Then
'One-time creation of event raiser. It used to be created in the
main
thread, but now it's created in
'the context of the timer's first-time-firing thread.
mEventRaiser = New FEventRaiser
End If

'As before...ask event raiser to raise event. This no longer works.
The
BeginInvoke in the RaiseAnEvent call
'does nothing.
mEventRaiser.RaiseAnEvent("This doesn't Work")

End Sub

End Class

I hope you can makes sense of this code, and any insight you can
provide
would be greatly appreciated.

 

[Mehdi]

'Force the creation of the form's handle.
While Not IsHandleCreated
Dim temp As IntPtr = Handle
End While

You don't need a while loop here. The handle will be created as soon as you
access the Handle property.

If I then modify this code somewhat so that the mEventRaiser is
actually
created once in the timer event handler, then it all stops working
(please
ignore the hack method I used for instantiating the CEventRaiser
object...the real way it's done is in a worker thread created at the
same [...]
'As before...ask event raiser to raise event. This no longer works.
The
BeginInvoke in the RaiseAnEvent call
'does nothing.
mEventRaiser.RaiseAnEvent("This doesn't Work")

A few points:

- Creating UI controls such as forms in another thread than the UI thread
is possible but dangerous and will generally create a lot more problems
than it solves. You should only do that if you have a very good
understanding of how the UI layer works in Windows. In 99.99% of the cases,
creating a form in another thread than the UI is the wrong solution.

- What exactly do you expect your BeginInvoke call to do when called on a
form created in another thread than the UI thread?

- The reason why BeginInvoke does nothing in your second case is probably
because it relies on posting messages to UI message queue to marhsall the
call to the UI thread but the thread on which your form has been created
doesn't have any message queue since you haven't called Application.Run()
in this thread nor shown a Form modally (which would create a message queue
and start a message pump automatically).

 

[Ian Mac]

'Force the creation of the form's handle.
While Not IsHandleCreated
Dim temp As IntPtr = Handle
End While

You don't need a while loop here. The handle will be created as soon as you
access the Handle property.

If I then modify this code somewhat so that the mEventRaiser is
actually
created once in the timer event handler, then it all stops working
(please
ignore the hack method I used for instantiating the CEventRaiser
object...the real way it's done is in a worker thread created at the
same [...]
'As before...ask event raiser to raise event. This no longer works.
The
BeginInvoke in the RaiseAnEvent call
'does nothing.
mEventRaiser.RaiseAnEvent("This doesn't Work")

A few points:

- Creating UI controls such as forms in another thread than the UI thread
is possible but dangerous and will generally create a lot more problems
than it solves. You should only do that if you have a very good
understanding of how the UI layer works in Windows. In 99.99% of the cases,
creating a form in another thread than the UI is the wrong solution.

- What exactly do you expect your BeginInvoke call to do when called on a
form created in another thread than the UI thread?

- The reason why BeginInvoke does nothing in your second case is probably
because it relies on posting messages to UI message queue to marhsall the
call to the UI thread but the thread on which your form has been created
doesn't have any message queue since you haven't called Application.Run()
in this thread nor shown a Form modally (which would create a message queue
and start a message pump automatically).

Thanks for the response, Mehdi.

In answer to your question regarding my expectation of the BeginInvoke
when called on a thread other than the UI thread, I would expect the
call to be placed in the message pump of the form that the BeginInvoke
is being called on, and subsequently processed in the thread that that
form was created on.

The big picture of what I'm trying to accomplish is to make a
component that can accept requests from any thread to raise an event
(this is the 'RaiseNewEvent' call in my sample code), but have the
component do the actual event raising on the thread that created the
component. I can't always guarentee that this component will be
created on a GUI thread (and in fact I have good reason in other code
for creating this component in a worker thread), so it still needs to
work regardless of what thread creates in.

In summary, if GUI thread 'A' spawns a worker thread 'B', and worker
thread 'B' instantiates an instance of my component, then if threads
'C', 'D', 'E', or whatever make calls to 'RaiseNewEvent', the actual
events MUST be raised on thread 'B' (the thread where the component
was initially created). I believe that there's absolutely no way to
achieve this type of functionality in .NET, but I'm hoping you or
someone can tell me otherwise.

Thanks again.

 

[Mehdi]

In summary, if GUI thread 'A' spawns a worker thread 'B', and worker
thread 'B' instantiates an instance of my component, then if threads
'C', 'D', 'E', or whatever make calls to 'RaiseNewEvent', the actual
events MUST be raised on thread 'B' (the thread where the component
was initially created). I believe that there's absolutely no way to
achieve this type of functionality in .NET, but I'm hoping you or
someone can tell me otherwise.

Right on top of my head, I effectively can't think of anything that would
fit your requirements. However, if you can modify your requirements
slightly, things become a lot easier:

Suggestion 1: implement a producer/consumer algorithm. For example, in the
constructor of your event raising component (which is called in thread B)
create a queue and start a new thread, say thread BB. In this new thread,
wait on an AutoResetEvent (that will put thread BB to sleep). Whenever
thread C, D or E wants an event to be raised, enqueue the event's data in
the queue then signal the AutoResetEvent. This will wake up thread BB which
can then dequeue the event's data and raise the event (you of course need
to synchronize access to the queue). All the events will therefore be
raised in thread BB (but not in thead B).

Suggestion 2: see if there is any valid reason why the events can not be
always raised in the UI thread. Things would be a lot easier if you could
simply require your component to be created in the UI thread then use the
method suggested in your original message to marshall all the events to the
UI thread.

Suggestion 3: see if there is any valid reason to require all the events to
be always raised in the same thread. Maybe you don't really need to go
through all this trouble and can simply raise the events in any thread.

 

[Ian Mac]

Right on top of my head, I effectively can't think of anything that would
fit your requirements. However, if you can modify your requirements
slightly, things become a lot easier:

Suggestion 1: implement a producer/consumer algorithm. For example, in the
constructor of your event raising component (which is called in thread B)
create a queue and start a new thread, say thread BB. In this new thread,
wait on an AutoResetEvent (that will put thread BB to sleep). Whenever
thread C, D or E wants an event to be raised, enqueue the event's data in
the queue then signal the AutoResetEvent. This will wake up thread BB which
can then dequeue the event's data and raise the event (you of course need
to synchronize access to the queue). All the events will therefore be
raised in thread BB (but not in thead B).

Suggestion 2: see if there is any valid reason why the events can not be
always raised in the UI thread. Things would be a lot easier if you could
simply require your component to be created in the UI thread then use the
method suggested in your original message to marshall all the events to the
UI thread.

Suggestion 3: see if there is any valid reason to require all the events to
be always raised in the same thread. Maybe you don't really need to go
through all this trouble and can simply raise the events in any thread.

Thanks Mehdi.

Suggestion 1 won't work because thread 'B' (where the component was
created) needs to be doing work other than just servicing the synch'ed
Q, and thus when it creates an instance of FEventRaiser, FEventRaiser
must NOT put the thread in a wait-state looking for AutoResetEvent
events. This was the whole reason for the event-raising component in
the first place...the user of the component can carry on doing what it
normally does, and only needs to respond to the FEventRaiser's
'MyEvent' event.

As for suggestions 2 and 3 , the reason I discovered the behavior I
originally posted was because I had a valid need to do exactly what I
posted. The code I posted has been in use for a couple of years, and
since nobody has ever tried instantiating the class in a thread other
than the GUI thread, it's always worked flawlessly, but a few days
ago, I tried creating an instance of a class (let's call it
CMyClass), that internally makes use of FEventRaiser, on a worker
thread, and the object failed to operate as it would/should had I
created it on the GUI thread. Since class CMyClass does some work that
can potentially block the thread it was created on for several seconds
(specifically, in the 'MyEvent' event raised from FEventRaiser),
instantiating CMyClass in the UI thread is no good.

I'd really like to know the technical reason why the message pump
behind FEventRaiser's form stops getting serviced simply because the
form was created in a worker thread. Perhaps if I knew this, it would
give me a clue as to how to make it work (either as is, or even taking
a completely different approach...as long as I can keep it black-box
and not break existing users of the component).

Thanks.

Ian

 

[Ian Mac]

Right on top of my head, I effectively can't think of anything that would
fit your requirements. However, if you can modify your requirements
slightly, things become a lot easier:

Suggestion 1: implement a producer/consumer algorithm. For example, in the
constructor of your event raising component (which is called in thread B)
create a queue and start a new thread, say thread BB. In this new thread,
wait on an AutoResetEvent (that will put thread BB to sleep). Whenever
thread C, D or E wants an event to be raised, enqueue the event's data in
the queue then signal the AutoResetEvent. This will wake up thread BB which
can then dequeue the event's data and raise the event (you of course need
to synchronize access to the queue). All the events will therefore be
raised in thread BB (but not in thead B).

Suggestion 2: see if there is any valid reason why the events can not be
always raised in the UI thread. Things would be a lot easier if you could
simply require your component to be created in the UI thread then use the
method suggested in your original message to marshall all the events to the
UI thread.

Suggestion 3: see if there is any valid reason to require all the events to
be always raised in the same thread. Maybe you don't really need to go
through all this trouble and can simply raise the events in any thread.

Thanks Mehdi.

Suggestion 1 won't work because thread 'B' (where the component was
created) needs to be doing work other than just servicing the synch'ed
Q, and thus when it creates an instance of FEventRaiser, FEventRaiser
must NOT put the thread in a wait-state looking for AutoResetEvent
events. This was the whole reason for the event-raising component in
the first place...the user of the component can carry on doing what it
normally does, and only needs to respond to the FEventRaiser's
'MyEvent' event.

As for suggestions 2 and 3 , the reason I discovered the behavior I
originally posted was because I had a valid need to do exactly what I
posted. The code I posted has been in use for a couple of years, and
since nobody has ever tried instantiating the class in a thread other
than the GUI thread, it's always worked flawlessly, but a few days
ago, I tried creating an instance of a class (let's call it
CMyClass), that internally makes use of FEventRaiser, on a worker
thread, and the object failed to operate as it would/should had I
created it on the GUI thread. Since class CMyClass does some work that
can potentially block the thread it was created on for several seconds
(specifically, in the 'MyEvent' event raised from FEventRaiser),
instantiating CMyClass in the UI thread is no good.

I'd really like to know the technical reason why the message pump
behind FEventRaiser's form stops getting serviced simply because the
form was created in a worker thread. Perhaps if I knew this, it would
give me a clue as to how to make it work (either as is, or even taking
a completely different approach...as long as I can keep it black-box
and not break existing users of the component).

Thanks.

Ian

 

[Mehdi]

Suggestion 1 won't work because thread 'B' (where the component was
created) needs to be doing work other than just servicing the synch'ed
Q,

Note that in suggestion 1, I suggested to create a new thread (thread BB)
to service the events, allowing thread B to keep doing its job normally. Of
course, creating a new thread might not work in your case if the existing
code you have assumes that events are raised in the thread where
FEventRaiser was created.

I'd really like to know the technical reason why the message pump
behind FEventRaiser's form stops getting serviced simply because the
form was created in a worker thread.

Because there is no message pump behind FEventRaiser's form in the first
place. Message pumps are not linked to Forms but to threads. A message pump
is nothing else than an infinite loop that keeps processing Windows
messages posted in its message queue by Windows, your application or other
applications. Creating a form does not create a message pump (if it did,
the Form's contructor would block forever since the message pump is an
infinite loop).

Whenever your application starts, you must be calling Application.Run() or
Form.ShowDialog() in the main thread (UI thread) of your application. These
2 methods create a message pump in the thread where they have been called.
These methods therefore block forever (until the form is closed in case of
ShowDialog() or until Application.Exit() is called in case of
Application.Run()). This message loop will be used to dispatch Windows
messages to any form created in this thread.

In your application, you are creating your form in a worker thread but you
are not creating a message pump in this worker thread (by calling
Application.Run or Form.ShowDialog). This means that your form doesn't have
any message pump that would allow it to receive Windows messages.
Control.Invoke() and Control.BeginInvoke() work by posting Windows message
to the message queue of the thread in which the form has been created (see
this article for details:
<http://weblogs.asp.net/justin_rogers/articles/126345.aspx>). They
therefore fail to work in your case since there is no message queue and no
message pump running on the thread in which you have created your form.

 

[Ian Mac]

Note that in suggestion 1, I suggested to create a new thread (thread BB)
to service the events, allowing thread B to keep doing its job normally. Of
course, creating a new thread might not work in your case if the existing
code you have assumes that events are raised in the thread where
FEventRaiser was created.


Because there is no message pump behind FEventRaiser's form in the first
place. Message pumps are not linked to Forms but to threads. A message pump
is nothing else than an infinite loop that keeps processing Windows
messages posted in its message queue by Windows, your application or other
applications. Creating a form does not create a message pump (if it did,
the Form's contructor would block forever since the message pump is an
infinite loop).

Whenever your application starts, you must be calling Application.Run() or
Form.ShowDialog() in the main thread (UI thread) of your application. These
2 methods create a message pump in the thread where they have been called.
These methods therefore block forever (until the form is closed in case of
ShowDialog() or until Application.Exit() is called in case of
Application.Run()). This message loop will be used to dispatch Windows
messages to any form created in this thread.

In your application, you are creating your form in a worker thread but you
are not creating a message pump in this worker thread (by calling
Application.Run or Form.ShowDialog). This means that your form doesn't have
any message pump that would allow it to receive Windows messages.
Control.Invoke() and Control.BeginInvoke() work by posting Windows message
to the message queue of the thread in which the form has been created (see
this article for details:
<http://weblogs.asp.net/justin_rogers/articles/126345.aspx>). They
therefore fail to work in your case since there is no message queue and no
message pump running on the thread in which you have created your form.

So that's what's going on. Thank-you for the explanation. The key to
my code not working is found in your statement "this message loop will
be used to dispatch Windows messages to any form created in this
thread." Would it be safe to say that an Invoke/BeginInvoke either
fails to post a message (thru SendMessage/PostMessage/
PostThreadMessage/etc?), or it does send/post OK, but the message pump
sees that the dest hWnd for the message was not created in its own
thread, and thus discards it?

An interesting observation about the non-working code is that when I
exit the application, several of the "lost" invocations DO get thru
(debug statements in the event handler let me see this). Typically, if
the code was running long enough for 20 or so BeginInvokes calls to be
made, when I shut down, I see 3 or 4 of the lost events actually fire.
Any idea what's going on with this?

 

[Mehdi]

Would it be safe to say that an Invoke/BeginInvoke either
fails to post a message (thru SendMessage/PostMessage/
PostThreadMessage/etc?), or it does send/post OK, but the message pump
sees that the dest hWnd for the message was not created in its own
thread, and thus discards it?

An interesting observation about the non-working code is that when I
exit the application, several of the "lost" invocations DO get thru
(debug statements in the event handler let me see this). Typically, if
the code was running long enough for 20 or so BeginInvokes calls to be
made, when I shut down, I see 3 or 4 of the lost events actually fire.
Any idea what's going on with this?

I can't really give you answers for that as I haven't dug that deep in the
Control.Invoke/BeginInvoke mechanism. The article I've mentionned in my
previous post should however provide you with at least part of the answers.
You can also use Reflector to have a look at how the
Control.Invoke/BeginInvoke mehods are implemented.

What's for sure however is that you won't be able to do what you're trying
to do. Not really because of limitations of .NET but because there is a
flaw in your logic.

Consider what the UI thread really is: it's just a normal thread that
happens to run an infinite loop (the message pump). This loop keeps looking
in the message queue to see if there are have been no new Windows messages
posted there. If there's a new message, it dequeues it and processes it. So
if the message is a button click message, it's gonna call the click event
handler method of the button (provided that you'll implemented it). If it's
a Paint message, it's gonna redraw the corresponding Form, calling the
Paint even handler in the process if it's been implemented. Whenever
another calls Control.Invoke or BeginInvoke to execute a method in the UI
thread, what happens is that a Windows message is posted in the message
queue. When the message loop dequeues this message, it executes the method
pointed by the delegate you passed to Control.Invoke/BeginInvoke. So in
effect, calling Control.Invoke/BeginInvoke does not interrupt the UI thread
in whatever it's doing to force it to execute your method but it simply
pushes a message in the message queue, message which is going to be
processed in the UI thread at some later time when all the Windows messages
that have been posted previously will have been processed.

Now, consider what a worker thread is: it's simply a thread that executes
its ThreadStart method in a procedural (linear) way. Once it's finished
executing its ThreadStart method, the thread dies. So there is no way you
can tell a worker thread: "Hey, hold on a second, stop whatever you're
doing and execute this method for me, will you". The only way you can
achieve what you want to do (execute events in a given worker thread) is to
enqueue any event you want your thread to execute in some sort of queue
then get your thread to periodically check the queue to see if there have
been new messages posted. So in effect, you need to implement a
producer/consumer algorithm.

 

[Ian Mac]

I can't really give you answers for that as I haven't dug that deep in the
Control.Invoke/BeginInvoke mechanism. The article I've mentionned in my
previous post should however provide you with at least part of the answers.
You can also use Reflector to have a look at how the
Control.Invoke/BeginInvoke mehods are implemented.

What's for sure however is that you won't be able to do what you're trying
to do. Not really because of limitations of .NET but because there is a
flaw in your logic.

Consider what the UI thread really is: it's just a normal thread that
happens to run an infinite loop (the message pump). This loop keeps looking
in the message queue to see if there are have been no new Windows messages
posted there. If there's a new message, it dequeues it and processes it. So
if the message is a button click message, it's gonna call the click event
handler method of the button (provided that you'll implemented it). If it's
a Paint message, it's gonna redraw the corresponding Form, calling the
Paint even handler in the process if it's been implemented. Whenever
another calls Control.Invoke or BeginInvoke to execute a method in the UI
thread, what happens is that a Windows message is posted in the message
queue. When the message loop dequeues this message, it executes the method
pointed by the delegate you passed to Control.Invoke/BeginInvoke. So in
effect, calling Control.Invoke/BeginInvoke does not interrupt the UI thread
in whatever it's doing to force it to execute your method but it simply
pushes a message in the message queue, message which is going to be
processed in the UI thread at some later time when all the Windows messages
that have been posted previously will have been processed.

Now, consider what a worker thread is: it's simply a thread that executes
its ThreadStart method in a procedural (linear) way. Once it's finished
executing its ThreadStart method, the thread dies. So there is no way you
can tell a worker thread: "Hey, hold on a second, stop whatever you're
doing and execute this method for me, will you". The only way you can
achieve what you want to do (execute events in a given worker thread) is to
enqueue any event you want your thread to execute in some sort of queue
then get your thread to periodically check the queue to see if there have
been new messages posted. So in effect, you need to implement a
producer/consumer algorithm.

I agree...I don't think it's doable.

And using a producer/consumer approach won't solve my problem either,
since the consumer would have to use a thread-blocking AutoResetEvent
event to look for new events to raise. To meet my needs, the thread
must NOT block.

Thanks for all the input, Mehdi. If I come up with a work-around, I'll
be sure to post it.