How to pump messages on a worker thread

[Mike23]

Hi there,

Can anyone fill me in on to how (manually) pump messages on a worker thread.
I'm calling a MSFT "ProcessAsync()" type of function which returns
immediately. The function takes a callback method which is invoked on
completion. I thought it invoked this callback on another thread but it's
calling it on the same thread I invoke "ProcessAsync()" from. Since
"ProcessAsync()" is called from my "Page_Load()" thread however (for those
familiar with ASP.NET), I probably need to launch my own worker thread and
have that invoke "ProcessAsync()" instead. The thread would then have to
pump messages so the callback can be invoked. Can anyone point me to the
function for doing this. Thanks in advance.

 

[Peter Duniho]

Hi there,

Can anyone fill me in on to how (manually) pump messages on a worker thread.
I'm calling a MSFT "ProcessAsync()" type of function which returns
immediately. The function takes a callback method which is invoked on
completion. I thought it invoked this callback on another thread but it's
calling it on the same thread I invoke "ProcessAsync()" from. Since
"ProcessAsync()" is called from my "Page_Load()" thread however (for those
familiar with ASP.NET), I probably need to launch my own worker thread and
have that invoke "ProcessAsync()" instead. The thread would then have to
pump messages so the callback can be invoked. Can anyone point me to the
function for doing this. Thanks in advance.

We just had a whole thread on the topic, including lots of (gradually
improving ) example code.

Note that whether it all applies to your scenario, I can't really say.
You seem to be using ASP.NET which I'm sure has its own specific
asynchronous paradigms and rules to follow. To that extent, you would
be much better off posting your question in a newsgroup specifically for
ASP.NET questions.

Whatever the "ProcessAsync()" method does, note that it may not
necessarily always execute the callback on the thread that call
ProcessAsync(). There may be some other rule, such as "any GUI thread"
or "any thread with a dispatcher" or something like that. In which
case, changing the thread where you _call_ ProcessAsync() isn't going to
help.

Callbacks should not be very complicated methods anyway, so it really
shouldn't be an issue for it to be executed on some particular thread.
But, if for some reason you feel you really need more control over where
the completion of the operation happens, you may find it makes more
sense to just use whatever the equivalent synchronous API is, and handle
the asynchronous aspect yourself.

Pete

 

[Mike23]

We just had a whole thread on the topic, including lots of (gradually

improving ) example code.

Note that whether it all applies to your scenario, I can't really say. You
seem to be using ASP.NET which I'm sure has its own specific asynchronous
paradigms and rules to follow. To that extent, you would be much better
off posting your question in a newsgroup specifically for ASP.NET
questions.

Whatever the "ProcessAsync()" method does, note that it may not
necessarily always execute the callback on the thread that call
ProcessAsync(). There may be some other rule, such as "any GUI thread" or
"any thread with a dispatcher" or something like that. In which case,
changing the thread where you _call_ ProcessAsync() isn't going to help.

Callbacks should not be very complicated methods anyway, so it really
shouldn't be an issue for it to be executed on some particular thread.
But, if for some reason you feel you really need more control over where
the completion of the operation happens, you may find it makes more sense
to just use whatever the equivalent synchronous API is, and handle the
asynchronous aspect yourself.

Thanks for the quick feedback (appreciated). Based on the behaviour, which
caught me by surprise, it appears very likely that the callback is
functioning like a single threaded apartment. I'm invoking "ProcessAsync()"
many times in a loop and I expected my callback to be invoked on another
thread. Instead, it's not being invoked at all until the "ProcessAsync()"
thread "exits". When it does, my breakpoint suddenly kicks in in the
callback. This behaviour indicates that "ProcessAsync()" is almost certainly
pumping messages back to the original thread. I have to keep that thread
alive however since it's an ASP.NET app (long story). So I'm now going to
launch my own worker thread where I'll call "ProcessAsync()". How do I pump
messages on this thread however, as well as post them from another thread?
Do you know the name of the functions off-hand or can you point me to the
"thread" you were referring to. Your help is appreciated. Thanks again.

 

[Mike23]

Thanks for the quick feedback (appreciated). Based on the behaviour, which

caught me by surprise, it appears very likely that the callback is
functioning like a single threaded apartment. I'm invoking
"ProcessAsync()" many times in a loop and I expected my callback to be
invoked on another thread. Instead, it's not being invoked at all until
the "ProcessAsync()" thread "exits". When it does, my breakpoint suddenly
kicks in in the callback. This behaviour indicates that "ProcessAsync()"
is almost certainly pumping messages back to the original thread. I have
to keep that thread alive however since it's an ASP.NET app (long story).
So I'm now going to launch my own worker thread where I'll call
"ProcessAsync()". How do I pump messages on this thread however, as well
as post them from another thread? Do you know the name of the functions
off-hand or can you point me to the "thread" you were referring to. Your
help is appreciated. Thanks again.

Ok, clearly I spoke way too soon. The callback thread is different after
all. Don't understand why it's held up until the main thread exits however.
Any ideas?

 

[Peter Duniho]

Thanks for the quick feedback (appreciated). Based on the behaviour, which
caught me by surprise, it appears very likely that the callback is
functioning like a single threaded apartment.

Just as long as you don't make any assumptions about it actually _being_
a COM STA implementation. Lots of things work similar to STA in COM,
but most are not actually STA in COM.

I'm invoking "ProcessAsync()"
many times in a loop and I expected my callback to be invoked on another
thread.

Note: without more information, no one here can confirm or refute your
expectation.

Instead, it's not being invoked at all until the "ProcessAsync()"
thread "exits".

What is "the 'ProcessAsync()' thread"? Again, lack of information from
you is hindering comprehension of your situation.

When it does, my breakpoint suddenly kicks in in the
callback. This behaviour indicates that "ProcessAsync()" is almost certainly
pumping messages back to the original thread.

Why does it indicate that? See above for comments about "lack of
information".

I have to keep that thread
alive however since it's an ASP.NET app (long story).

Story long or short, as I mentioned before, if you're writing an ASP.NET
app and this is part of the issue, you should probably be limiting your
question to the ASP.NET newsgroup. Otherwise, you'll get advice from us
non-ASP.NET folks that may just confuse things (because it leaves out
important ASP.NET-relevant information).

Conversely, if your question isn't related to ASP.NET, you should be
able to produce a concise-but-complete code example that doesn't use
ASP.NET.

So I'm now going to
launch my own worker thread where I'll call "ProcessAsync()". How do I pump
messages on this thread however, as well as post them from another thread?
Do you know the name of the functions off-hand or can you point me to the
"thread" you were referring to. Your help is appreciated. Thanks again.

The other message thread in question has the subject "Checking for GUI
overload". There are other examples; that just happens to be the most
recent. In general, you are looking for discussions that involve
queues, delegates, and invocation.

Pete

 

[Peter Duniho]

Ok, clearly I spoke way too soon. The callback thread is different after
all. Don't understand why it's held up until the main thread exits however.
Any ideas?

Without a concise-but-complete code example, no. None at all. We don't
even know what the "ProcessAsync()" method is supposed to do, never mind
details such as how it's implemented or why that implementation might
not signal completion until some particular thread exits.

One possible example: perhaps you have some kind of almost-deadlock,
where the asynchronous operations _can't_ complete until the main thread
has exited. But again, without more information, refining that example
to something that actually _explains_ what you're seeing, as well as how
to deal with it, is going to be practically impossible.

Pete

 

[Mike23]

Just as long as you don't make any assumptions about it actually _being_ a

COM STA implementation. Lots of things work similar to STA in COM, but
most are not actually STA in COM.

I'm aware of the details

What is "the 'ProcessAsync()' thread"? Again, lack of information from
you is hindering comprehension of your situation.

Trying to keep this simple (I cross-posted to ASP.NET BTW but it may be a
simple threading issue which is why I posted to C# as well). There's a
well-known ASP.NET method called "Page_Load()" that every ASP.NET developer
is familiar with it. It's invoked by the system (on a web server) when a web
page is requested (to load the requested page). On this thread I'm calling
"System.Web.Services.Protocols.SoapHttpClientProtocol.InvokeAsync()" which I
just discovered after doing some digging around (the class I'm using is from
MSFT - it's a derivative of the latter class). The callback is being passed
via the 3rd arg to this function. Once I pass it, the function returns
immediately. I expected my callback to be invoked on another thread at this
point (it's thread safe) but not the same thread where "InvokeAsync()" is
called from. That thread is blocked instead waiting for the callback to
complete its work. I have a "ManualResetEvent" set up that the callback
thread will signal when done. I'll have to take a closer look at this
"InvokeAsync()" method to see what it really does.

Why does it indicate that? See above for comments about "lack of
information".

It's indicated because that's the way an STA thread works
If other messages are waiting in the thread's message queue, they won't be
picked up and dispatched until the thread pulls those messages off the
queue. That normally doesn't happen until the thread returns to the top of
the call stack (usually) to pick up another message (standard Windows
message pumping architecture). It appeared at first that's what was
happening but apparently not.

Story long or short, as I mentioned before, if you're writing an ASP.NET
app and this is part of the issue, you should probably be limiting your
question to the ASP.NET newsgroup. Otherwise, you'll get advice from us
non-ASP.NET folks that may just confuse things (because it leaves out
important ASP.NET-relevant information).

Agreed, it could very well be some ASP.NET issue. But a thread is a thread
so it's unclear what the problem is.

Conversely, if your question isn't related to ASP.NET, you should be able
to produce a concise-but-complete code example that doesn't use ASP.NET.

Yes, and ultimately I'll have to try that. I wanted to avoid it because it's
difficult to get everything set up outside the current environment. Now that
I know the underlying method however (previously mentioned), hoepfully I can
pin the problem down.

The other message thread in question has the subject "Checking for GUI
overload". There are other examples; that just happens to be the most
recent. In general, you are looking for discussions that involve queues,
delegates, and invocation.

Thanks very much.

 

[Mike23]

Conversely, if your question isn't related to ASP.NET, you should be able
Interesting enough, I just tested the same code in a console app outside of
ASP.NET. It behaves as expected. So it appears there may be an ASP.NET issue
after all. These insidiuous differences with no easty-to-find explanation
are enough to make me want to change careers.

 

[Peter Duniho]

I'm aware of the details


Trying to keep this simple (I cross-posted to ASP.NET BTW but it may be a
simple threading issue which is why I posted to C# as well). There's a
well-known ASP.NET method called "Page_Load()" that every ASP.NET developer
is familiar with it. It's invoked by the system (on a web server) when a web
page is requested (to load the requested page). On this thread I'm calling
"System.Web.Services.Protocols.SoapHttpClientProtocol.InvokeAsync()" which I
just discovered after doing some digging around (the class I'm using is from
MSFT - it's a derivative of the latter class).

Unfortunately, the documentation for
SoapHttpClientProtocol.InvokeAsync() is almost non-existent. However,
it definitely looks very ASP.NET-specific. Browsing through the class
information, I'm not even convinced that InvokeAsync() is strictly a
threading-related method, as it appears to relate to client/server
invocation (i.e. invocation of a method via proxy).

The callback is being passed
via the 3rd arg to this function. Once I pass it, the function returns
immediately. I expected my callback to be invoked on another thread at this
point (it's thread safe) but not the same thread where "InvokeAsync()" is
called from.

What gave you that expectation?

That thread is blocked instead waiting for the callback to
complete its work.

What is the point of using an asynchronous method, only to sit and block
until it's done? Why not just call the synchronous version instead?

Note that in other contexts, doing that sort of thing can cause a _real_
deadlock. For example, in a Forms application, if you call
Control.BeginInvoke() from the owning thread, and then immediately after
the call block on something like ManualResetEvent.WaitOne(), your
program will deadlock.

Sounds like you might be getting off lucky, just having your callbacks
delayed, rather than blocked completely.

I have a "ManualResetEvent" set up that the callback
thread will signal when done. I'll have to take a closer look at this
"InvokeAsync()" method to see what it really does.

That sounds like a good idea to me. I think it's always a very good
idea to not write code that calls a method until you know what the
method really does.

It's indicated because that's the way an STA thread works

Well, you should see the immediate fallacy in that conclusion then.

[...]
Agreed, it could very well be some ASP.NET issue. But a thread is a thread
so it's unclear what the problem is.

But that's not true. If you're writing an ASP.NET program, it's not
strictly true that "a thread is a thread". ASP.NET has all sorts of
extra client/server proxying/marshaling/messaging/whatever that does
exist in a normal single-process, single-computer implementation.

Sounds to me like the starting point here is to find something that does
a good job describing what the InvokeAsync() method really does and why
a program would use it. I suspect it doesn't do what you seem to think
it does, at least not precisely (one hopes it has _something_ to do with
asynchronous invocation of code, but there are lots of different ways
that can happen...you can't just assume one particular implementation).

Pete

 

[Mike23]

Unfortunately, the documentation for SoapHttpClientProtocol.InvokeAsync()

is almost non-existent.

Yes, this and so many others unfortunately.

What gave you that expectation?

It's impractical for it to work any other way (designing it to do so would
be foolish)

What is the point of using an asynchronous method, only to sit and block
until it's done? Why not just call the synchronous version instead?

I'm calling a (remote) MSFT web service potentially hundreds or thousands of
times. While those requests are being serviced, the calling thread can carry
on its work and pick up the results at a later point (the callback
accumulates these results as each service request returns). This
significantly improves performance opposed to making hundredes of
synchronous calls (holding up the main thread as it waits for each to
return)

Note that in other contexts, doing that sort of thing can cause a _real_
deadlock. For example, in a Forms application, if you call
Control.BeginInvoke() from the owning thread, and then immediately after
the call block on something like ManualResetEvent.WaitOne(), your program
will deadlock.

Sounds like you might be getting off lucky, just having your callbacks
delayed, rather than blocked completely.

They shouldn't delay for the length of time involved however. Something's
not right. Moreover, I do need to block the main thread when it's finished
its work. It needs to block until the callback has finished receiving the
responses from each of the original service requests. The callback is never
called however unless I let the main thread exit. Not good.

Well, you should see the immediate fallacy in that conclusion then.

I think we're probably off-topic now though. I do know my way around these
issues (maybe it doesn't appear that way) but I'm not an ASP.NET expert per
se. There appears to be some esoteric threading issue in that arena since
the code works correctly from a standalone console app (just tested it).

But that's not true. If you're writing an ASP.NET program, it's not
strictly true that "a thread is a thread". ASP.NET has all sorts of extra
client/server proxying/marshaling/messaging/whatever that does exist in a
normal single-process, single-computer implementation.

I agree and this may be an example. Usually though, it's not as complex as
it seems. Things tend to work in a WYSIWYG fashion for the most part.

Sounds to me like the starting point here is to find something that does a
good job describing what the InvokeAsync() method really does and why a
program would use it. I suspect it doesn't do what you seem to think it
does, at least not precisely (one hopes it has _something_ to do with
asynchronous invocation of code, but there are lots of different ways that
can happen...you can't just assume one particular implementation).

Unfortunately it's a MSFT class I'm relying on and that's what's calling the
above function behind the scenes. There's no other alternative however (the
class is downloaded from MSFT's Virtual Earth Services).

 

[J.B. Moreno]

I think we're probably off-topic now though. I do know my way around these
issues (maybe it doesn't appear that way) but I'm not an ASP.NET expert per
se. There appears to be some esoteric threading issue in that arena since
the code works correctly from a standalone console app (just tested it).

Did you first extract the code that you're concerned with into an
ASP.NET page, test it, and then turn that into a console app? Or just
extract the code into a console app?