Threading problem with Garbage Collector

[Christian Kaiser]

We have a component that has no window. Well, no window in managed code - it
uses a DLL which itself uses a window, and this is our problem!

When the garbage collector runs and removes our component (created
dynamically by, say, a button click, and then not referenced any more), the
GC runs in a different thread, which prohibits the DLL to destroy its
window, resulting in a GPF when the WndProc of that window is called - the
code is gone (DLL unloaded), but the window still there....

Clear to see (took us hours, to be honest, to understand what caused the
GPF, as WinDBG somehow uses 100% CPU load if used with .NET 2.0), which was
a PITA and a lot of guesswork.

Is there a way to get around that problem, so that we can call a routine
from Dispose() in the context of the original thread?

We tried creating a window ("m_myLabel = new Label()") in the component's
constructor, and use a delegate and "m_myLabel.Invoke(...)" this to get a
synchroneous invoke in the original thread. Problem is, the component has no
form to create the "Label" window with, thus invoke throws an exception
telling us that a non-existing window does not support Invoke. Right it is.

Questions:

a) How can a window be created without a parent form (invisible,
independent, just for this delegate), or

b) is there a better way to tell the GC to use the original "main" thread?

Christian

 

[Willy Denoyette [MVP]]

| We have a component that has no window. Well, no window in managed code -
it
| uses a DLL which itself uses a window, and this is our problem!
|

Components and DLL's are such general terms that say so little about what
they actually are, so you will have to be more explicit.
What's exactly the component? I guess it's a managed class.
What's contained in the DLL, how do you access the methods/functions?
Through PInvoke or COM interop?
If COM
- what are it's threading requiremnets (apartment type)?
- What is the apartment state of the thread on which it is created?

| When the garbage collector runs and removes our component (created
| dynamically by, say, a button click, and then not referenced any more),
the
| GC runs in a different thread, which prohibits the DLL to destroy its
| window, resulting in a GPF when the WndProc of that window is called - the
| code is gone (DLL unloaded), but the window still there....
|

Hmmm... who's unloading the DLL? The GC doesn't unload DLL's.


| Clear to see (took us hours, to be honest, to understand what caused the
| GPF, as WinDBG somehow uses 100% CPU load if used with .NET 2.0), which
was
| a PITA and a lot of guesswork.
|
| Is there a way to get around that problem, so that we can call a routine
| from Dispose() in the context of the original thread?
|
| We tried creating a window ("m_myLabel = new Label()") in the component's
| constructor, and use a delegate and "m_myLabel.Invoke(...)" this to get a
| synchroneous invoke in the original thread. Problem is, the component has
no
| form to create the "Label" window with, thus invoke throws an exception
| telling us that a non-existing window does not support Invoke. Right it
is.
|
| Questions:
|
| a) How can a window be created without a parent form (invisible,
| independent, just for this delegate), or
|

You can't, and it's not needed, your DLL code creates the window so it's
responsible for it's life time.

| b) is there a better way to tell the GC to use the original "main" thread?
|

No, and this is not the problem anyway, the GC doesn't know about unmanaged
stuff.
I guess your problem relates to threading issues with COM interop, but I
could be wrong of course.

Willy.

 

[Christian Kaiser]

Hi Willy,


OK, sorry, back to square one, I'll try to be more specific ;-)

The component is a managed C# control. It itself controls calls to a DLL
(written in Delphi, but that itself is not the basic problem) via PInvoke.
So the C# class is meant to be a proxy to the DLL's functions, some more or
less slim wrapper around its functions.

When the component is created (in the main thread, let's call it thread
"A"), it loads the DLL, which in turn then creates a window (it needs to,
unfortunately). It does so right after LoadLibrary of the DLL, in the
startup code of the DLL (while initializing static variables, ...).

The problem comes when the component is disposed of by the Garbage
Collector, which has a different thread "B". The Dispose() of the component
unloads the DLL, which tries to destroy its window. And this now is the
problem - the DLL called by thread "B" cannot destroy the window created in
thread "A". Here it's the problem of Delphi, I assume, but the problem is
more or less valid in other languages too. DestroyWindow() does not work,
sendin a WM_CLOSE still lets the window survive, ... -> GPF.

The solution we thought about then was to create a window in our component
(using a "Label" for example) to be able to switch from thread "B" to thread
"A" in a standard way - synchronous access to a control's functions. So in
its constructor, the component wants to create the label window, and in the
Dispose() method it uses the label window to synchronize the FreeLibrary()
by invoking a delegate, which calls an event of the label, which in turn (as
it's in thread A by then) can free the DLL as it should be. Draft (from
memory, I'm at home now):

delegate void DelegateKillDLL();

class component
{
void KillDLL()
{
// Free external DLL...
}

component()
{
myLabel = new Label();
// Load external DLL.
}

void Dispose()
{
DelegateKillDLL dlg = new DelegateKillDLL(KillDLL);
myLabel.Invoke(dlg, object[] {});
}
}

a) the Invoke() call does not work, as "new Label()" did not create a
window. So "Invoke()" throws an exception that it cannot work without any
window. Is there a way to create a window for such a purpose (WS_OVERLAPPED
style, so to speak)? Our component is not a "visual" control (it does not
have any window, is not derived from a visual component, the "Container"
property is null).

b) will the whole idea work in a GC thread?

c) Is there a better way to do this?

I hope I was able to describe it better than before. If there's any
question, don't hesitate to ask. I'm relatively new to C# (but very
experienced in C++, Win32, ...)


Christian

 

[Alan Pretre]

When the garbage collector runs and removes our component (created
dynamically by, say, a button click, and then not referenced any more),
the
GC runs in a different thread, which prohibits the DLL to destroy its
window, resulting in a GPF when the WndProc of that window is called - the
code is gone (DLL unloaded), but the window still there....

Couple of things I'm wondering,

Are you using Dispose()? Maybe you could dispose on the thread that loaded
the DLL, then prevent the dispose if happening from GC
(GC.SuppressFinalize).

Or, maybe you could load stuff using a different appdomain, then discard the
appdomain later?

-- Alan

 

[Willy Denoyette [MVP]]

| Hi Willy,
|
|
| OK, sorry, back to square one, I'll try to be more specific ;-)
|
| The component is a managed C# control. It itself controls calls to a DLL
| (written in Delphi, but that itself is not the basic problem) via PInvoke.
| So the C# class is meant to be a proxy to the DLL's functions, some more
or
| less slim wrapper around its functions.
|
| When the component is created (in the main thread, let's call it thread
| "A"), it loads the DLL, which in turn then creates a window (it needs to,
| unfortunately). It does so right after LoadLibrary of the DLL, in the
| startup code of the DLL (while initializing static variables, ...).
|
| The problem comes when the component is disposed of by the Garbage
| Collector, which has a different thread "B". The Dispose() of the
component
| unloads the DLL, which tries to destroy its window. And this now is the
| problem - the DLL called by thread "B" cannot destroy the window created
in
| thread "A". Here it's the problem of Delphi, I assume, but the problem is
| more or less valid in other languages too. DestroyWindow() does not work,
| sendin a WM_CLOSE still lets the window survive, ... -> GPF.
|

Ok, here I assume the component (the control) implements Dispose() (or the
disposable pattern), but you don't call Dispose explicitely from the control
thread, instead you wait until the finalizer runs.
Calling Dispose from the destructor (finalizer) fails, because DestroyWindow
MUST be called from the HWND owning thread, which is the thread that created
the control. The finalizer always runs on a separate thread so it will
always fail to destroy HWND's from this thread.
The most obvious solution, is to explicitely call Dispose from the thread
that created the control (or use the "using" idiom) when you are done with
the control.
If applying the using idiom is not feasible, you need to marshal the call
(using Control.Invoke) to the thread that created the control (I assume the
UI thread) and call DestroyWindow and unload the DLL in this thread.

Willy.

 

[Christian Kaiser]

Willy,

Ok, here I assume the component (the control) implements Dispose() (or the
disposable pattern), but you don't call Dispose explicitely from the
control
thread, instead you wait until the finalizer runs.
Calling Dispose from the destructor (finalizer) fails, because
DestroyWindow
MUST be called from the HWND owning thread, which is the thread that
created
the control. The finalizer always runs on a separate thread so it will
always fail to destroy HWND's from this thread.
The most obvious solution, is to explicitely call Dispose from the thread
that created the control (or use the "using" idiom) when you are done with
the control.
If applying the using idiom is not feasible, you need to marshal the call
(using Control.Invoke) to the thread that created the control (I assume
the
UI thread) and call DestroyWindow and unload the DLL in this thread.

exactly.



If I would accept that customers call Dispose() in the main thread, this
would be fine, but I don't like the idea that they get a GPF when they don't
read the documentation (which is very likely today). So is possible I would
like not to use this (also, while Dispose()ing here, we get a LoaderLock
violation in VS2005, but that's another thing.

The marshalling you mention is exactly what I would like to do, and that was
my main question: how can I create a window in my component (not derived
from Control, but - as it's a non-UI-component - from Component, thus I have
no Container that I can put the window in) that I can use for marshalling?
This was my first problem, as constructing a "Label" instance (as
example/first test) seems not to create a window, thus Invoke() failed.

Imagine:

class MyComponent : Component
{

private label myLabel;


component()
{
myLabel = new Label();
// Load external DLL.
}

void Dispose()
{
KillDLL();
}


void KillDLL()
{
if (myLabel.InvokeRequired)
{
DelegateKillDLL dlg = new DelegateKillDLL(KillDLL);
myLabel.Invoke(dlg, object[] {});

GC.SuppressFinalize();
}
else
{
// Free external DLL...
}
}


}

(Outlook Express kills all formatting, sorry for that).

Here, the InvokeRequired already tells me NO, even if the function is called
from the GC thread! I wonder why - maybe myLabel window has no window at
all? If I ignore the InvokeRequired return value, I get
"System.InvalidOperationException: Invoke or BeginInvoke cannot be called on
a control until the window handle has been created".


Christian

 

[Christian Kaiser]

Alan,

thanks for getting back at me.

We offer the component for customers to use, so we would have to force them
to use Dispose() if they create it dynamically (new OurComponent()). If used
in the IDE (component included in the xxx.design.cs), it works - but you
know customers always do things differently than you expect ;-)

Please look at my other response for more details of my problems.

AppDomain is out as far as I know, as I need the other DLL in the main UI
thread (for complicated internal reasons). In addition, this would either
require us to deliver another DLL, or the customers to deal with AppDomains,
which is not everyone's daily work.

Christian

 

[Willy Denoyette [MVP]]

| Willy,
|
| > Ok, here I assume the component (the control) implements Dispose() (or
the
| > disposable pattern), but you don't call Dispose explicitely from the
| > control
| > thread, instead you wait until the finalizer runs.
| > Calling Dispose from the destructor (finalizer) fails, because
| > DestroyWindow
| > MUST be called from the HWND owning thread, which is the thread that
| > created
| > the control. The finalizer always runs on a separate thread so it will
| > always fail to destroy HWND's from this thread.
| > The most obvious solution, is to explicitely call Dispose from the
thread
| > that created the control (or use the "using" idiom) when you are done
with
| > the control.
| > If applying the using idiom is not feasible, you need to marshal the
call
| > (using Control.Invoke) to the thread that created the control (I assume
| > the
| > UI thread) and call DestroyWindow and unload the DLL in this thread.
|
|
| exactly.
|
|
|
| If I would accept that customers call Dispose() in the main thread, this
| would be fine, but I don't like the idea that they get a GPF when they
don't
| read the documentation (which is very likely today). So is possible I
would
| like not to use this (also, while Dispose()ing here, we get a LoaderLock
| violation in VS2005, but that's another thing.
|


A customer that fails to call Dispose when applicable introduces a bug in
his code, one way to solve this is by throwing an exception in the
finalizer, this will force him to correct his code and read the docs. I'm
not sure about the other issue, but it's strange that you get a LoaderLock
issue when calling Dispose, but not when the finalizer calls dispose, a bug
you will have to correct anyway once the users start to read the docs ;-).


| The marshalling you mention is exactly what I would like to do, and that
was
| my main question: how can I create a window in my component (not derived
| from Control, but - as it's a non-UI-component - from Component, thus I
have
| no Container that I can put the window in) that I can use for marshalling?
| This was my first problem, as constructing a "Label" instance (as
| example/first test) seems not to create a window, thus Invoke() failed.
|
| Imagine:
|
| class MyComponent : Component
| {
|
| private label myLabel;
|
|
| component()
| {
| myLabel = new Label();
| // Load external DLL.
| }
|
| void Dispose()
| {
| KillDLL();
| }
|
|
| void KillDLL()
| {
| if (myLabel.InvokeRequired)
| {
| DelegateKillDLL dlg = new DelegateKillDLL(KillDLL);
| myLabel.Invoke(dlg, object[] {});
|
| GC.SuppressFinalize();
| }
| else
| {
| // Free external DLL...
| }
| }
|
|
| }
|
| (Outlook Express kills all formatting, sorry for that).
|
| Here, the InvokeRequired already tells me NO, even if the function is
called
| from the GC thread! I wonder why - maybe myLabel window has no window at
| all? If I ignore the InvokeRequired return value, I get
| "System.InvalidOperationException: Invoke or BeginInvoke cannot be called
on
| a control until the window handle has been created".
|

But you do have a main form isn't it? (Please correct me if I'm wrong). If
you do have a form all you have to do is marshal the delegate to this form
(which is a control). The message pump (of the thread that created the form,
which is the thread on which you created the component, right?) will pick-up
the message and execute the callback on the same thread.
Something like this will do:

void KillDLL()
{
if (myForm.InvokeRequired)
{
DelegateKillDLL dlg = new DelegateKillDLL(KillDLL);
myForm.Invoke(dlg, object[] {});

GC.SuppressFinalize();
}
else
{
// Free external DLL...
}
}

Willy.

 

[Christian Kaiser]

Thanks for getting back,

A customer that fails to call Dispose when applicable introduces a bug in
his code, one way to solve this is by throwing an exception in the
finalizer, this will force him to correct his code and read the docs. I'm
not sure about the other issue, but it's strange that you get a LoaderLock
issue when calling Dispose, but not when the finalizer calls dispose, a
bug
you will have to correct anyway once the users start to read the docs ;-).

Yep, but one after the other, please ;-)

But you do have a main form isn't it? (Please correct me if I'm wrong). If
you do have a form all you have to do is marshal the delegate to this form
(which is a control). The message pump (of the thread that created the
form,
which is the thread on which you created the component, right?) will
pick-up
the message and execute the callback on the same thread.

No, I don't have a form. We offer customers our component. They can use it
with whatever they like - a non-UI application without any form, a web page,
whatever, as long as they can instantiate it. Thus we need a non-modal,
overlapped, parentless, non-child window for the marshalling. A window just
like one that any Win32/64 code can create with CreateWindow().

Thus our component needs to be able to create a single, simple, independent
window which still will be used to synchronize the access to the UI thread,
as its messages are also being dispatched from the main message loop. Is
that impossible in C#?



Christian

 

[Willy Denoyette [MVP]]

| Thanks for getting back,
|
| > A customer that fails to call Dispose when applicable introduces a bug
in
| > his code, one way to solve this is by throwing an exception in the
| > finalizer, this will force him to correct his code and read the docs.
I'm
| > not sure about the other issue, but it's strange that you get a
LoaderLock
| > issue when calling Dispose, but not when the finalizer calls dispose, a
| > bug
| > you will have to correct anyway once the users start to read the docs
;-).
|
| Yep, but one after the other, please ;-)
|
|
|
| > But you do have a main form isn't it? (Please correct me if I'm wrong).
If
| > you do have a form all you have to do is marshal the delegate to this
form
| > (which is a control). The message pump (of the thread that created the
| > form,
| > which is the thread on which you created the component, right?) will
| > pick-up
| > the message and execute the callback on the same thread.
|
|
|
| No, I don't have a form. We offer customers our component. They can use it
| with whatever they like - a non-UI application without any form, a web
page,
| whatever, as long as they can instantiate it. Thus we need a non-modal,
| overlapped, parentless, non-child window for the marshalling. A window
just
| like one that any Win32/64 code can create with CreateWindow().
|

That means you have no control whatever over the thread your component runs
on, right? That means that if the user blocks the message pump, or if he
uses your component on different threads simultaneously, you are in trouble.
The reason is that you rely on the finalizer to clean-up thread affinitized
unmanaged resources, something you should try to workaround, realy.


| Thus our component needs to be able to create a single, simple,
independent
| window which still will be used to synchronize the access to the UI
thread,
| as its messages are also being dispatched from the main message loop. Is
| that impossible in C#?
|

One way is to create a simple 'hidden' form and run a message loop in your
component, note that you'll have to close the form when done with it, and
you need to start this in your main thread.


HiddenForm msr = new HiddenForm();
....

// Need to derive from ApplicationContext
class HiddenForm : ApplicationContext
{
Form form1;
public HiddenForm()
{
form1 = new Form();
Application.Run(this); // start the message pump on the callers thread
which becomes the UI thread!
}
...
}



Willy.

 

[Christian Kaiser]

OK, the threading issue is very important... thank you very much for that
consideration!

You say: the HiddenForm you suggest is needed in a component used in a
different, non-UI thread (makes sense - no pump there...).

Problems I see: what happens if
- the thread already has a message pump somewhere in the user's thread code?
We may not call Application.Run then.
- the component, which is used in the main thread, uses the code you
suggested?

AFAIK, it's impossible to check these "states" (am I in a UI thread, ...)
reliably.

So you are correct with your statement that we need to rely on the user to
Dispose() the component in the correct thread, and forget all about the
garbage collector - maybe issuing a message box if the GC tries to get rid
of the component.

Sigh. You know how good customers read any documentation...

Christian

 

[bughunter]

So you are correct with your statement that we need to rely on the user to
Dispose() the component in the correct thread, and forget all about the
garbage collector - maybe issuing a message box if the GC tries to get rid
of the component.

Sigh. You know how good customers read any documentation...

I agree that relying on users of the DLL calling Dispose isn't good.
Throwing an exception from teh finalizer to alert developers that they
shoul dbe calling dispose seems like a bad idea to. The GC can fire at
any time and an old object could be in GC generation 1 or 2, thus the
finalizer may get called a long time after the object has gone out of
scope.

Therefore you may not get the exception during normal testing, release
a binary and then get customer reports in of exceptions occuring at
random times.

Perhaps a slightly better approach would be to throw the exception only
when in debug mode using an #if DEBUG directive. This is far from ideal
though, and I'd be interested to know if there is an official Microsoft
response to this scenario.

Regards,

Colin

 

[Brian Gideon]

Christian,

I had a similar problem. What I did was capture the thread id used to
create the window and then when the GC called Dispose I used that
thread id in a call to PostThreadMessage to send a message to the
correct thread. I then captured the message in the target thread via
an IMessageFilter that I added by calling Application.AddMessageFilter.
I'm not sure of the exact details in your situation, but you may be
able to do something similar.

You can create a message only window by setting the parent window to
HWND_MESSAGE.

Brian

 

[Willy Denoyette [MVP]]

|
| Christian Kaiser wrote:
| > So you are correct with your statement that we need to rely on the user
to
| > Dispose() the component in the correct thread, and forget all about the
| > garbage collector - maybe issuing a message box if the GC tries to get
rid
| > of the component.
| >
| > Sigh. You know how good customers read any documentation...
|
| I agree that relying on users of the DLL calling Dispose isn't good.
| Throwing an exception from teh finalizer to alert developers that they
| shoul dbe calling dispose seems like a bad idea to. The GC can fire at
| any time and an old object could be in GC generation 1 or 2, thus the
| finalizer may get called a long time after the object has gone out of
| scope.
|
| Therefore you may not get the exception during normal testing, release
| a binary and then get customer reports in of exceptions occuring at
| random times.
|
| Perhaps a slightly better approach would be to throw the exception only
| when in debug mode using an #if DEBUG directive. This is far from ideal
| though, and I'd be interested to know if there is an official Microsoft
| response to this scenario.
|
| Regards,
|
| Colin
|

Did you actually read the whole topic? The problem here is that the
finalizer thread runs in the wrong apartment, relying on it to clean-up
unmanaged resources doesn't even work in this scenario, so the only thing
you can do is throw or change your design. V2 offers a number of tools to
get rid of the finalizer, something you might consider if you want to write
reliable code, something that is not possible when relying on finalizers to
run, they might run at all while you think they run late.

Willy.

 

[bughunter]

Hi,

Did you actually read the whole topic?

I thought I had.

The problem here is that the
finalizer thread runs in the wrong apartment, relying on it to clean-up
unmanaged resources doesn't even work in this scenario,

Yes this is the problem I had that led me to this discussion. I
discovered a cleanup routine that could be called either from dispose
or from a finalizer. The routine was releasing window handles and I
wondered if that should really only be done from the UI thread (due to
thread affinity) that created the handle - rather than the finalizer
thread.

so the only thing
you can do is throw or change your design.

Sure, but it looks like a choice between switching threads within a
finalizer - by which I mean calling Invoke or BeginInvoke - or ensuring
the user of a class calls a cleanup routine (Dispose) which we can't
force them to do. And switching threads from a finalizer feels like it
could a source of problems.

V2 offers a number of tools to
get rid of the finalizer, something you might consider if you want to write
reliable code,

ok, are there any articles/blogs/docs you could point us to for
starters? e.g. are you referring to the constrained execution regions?

something that is not possible when relying on finalizers to
run, they might run at all while you think they run late.

Part of the problem - and I have been guilty of this to some degree -
is not fully understanding when and how they should be used. I've
traced a couple of bugs to finalizers recently and they were hard to
isolate due to the seemingly random nature of their occurance. In one
case the solution was simply to remove the finalizer, it just wasn't
needed, and I suspect this might be a common scenario.

Colin.