Illegal Instruction Thrown When Executing a Callback Function

[bsnguy]

I have two callback functions declared in my C# code:

public unsafe static void CallbackA()
/* yes, I do mean to declare this as unsafe and static in my real code...
read on */
{
MessageBox.Show(@"Debug.", @"Got Here A");
}

public void CallbackB()
{
MessageBox.Show(@"Debug.", @"Got Here B");
}

I assign them delegates and pass the pointers to my DLL (which is written in
unmanaged C code). In my C code, I use them like this:

void (CALLBACK *CallbackA)(void) = NULL;
void (CALLBACK *CallbackB)(void) = NULL;

int FunctionA (void) {
...
CallbackA();
...
}

void FunctionB (void) {
...
CallbackB();
...
}

in other words, things are pretty much the same except the return value.

- Every time I call FunctionB which uses CallbackB, all is fine (Callback B
runs as expected).

- When I call CallbackA in FunctionA, the very first time, it crashes with
an Illegal Instruction exception.

- if I call CallbackA from within FunctionB, it works just fine.

- if I call CallbackB from within FunctionA, it fails.

- When I ran the same DLL with unmanaged C code, all worked fine and as
expected.

So, it's something about FunctionA's return parameter that's causing this?
Doesn't make sense to me, but it sure looks that way.

Any ideas on what else to look for or how I might fix this?

I figure that the root cause is that the function pointer is bad and I'm
trying to execute data (the stack, heap, etc.). I cannot explain why, though.
When I look at the pointers passed, they look right.

 

[john]

How did you access the global function pointer in the C dll from you c# app?
P/Inoke?

[...]
- When I call CallbackA in FunctionA, the very first time, it crashes
with
an Illegal Instruction exception.

Does it crash before executing CallbackA() or after?

Assuming this guess is correct...

[...]
So, it's something about FunctionA's return parameter that's causing
this?
Doesn't make sense to me, but it sure looks that way.

...one possible explanation is that you've got an incorrect calling
convention. I don't know off the top of my head what requirement, if any,
.NET has with respect to calling convention. But given that at least one
significant calling convention difference will only show up in functions
that return values, that seems like a likely culprit to me. It certainly
could produce an illegal instruction exception, at least when trying to
return from FunctionA().

Off the top of my head, I'd guess .NET uses STDCALL, but you really ought
to just look it up and make sure.

Pete

 

[bsnguy]

[...]
- When I call CallbackA in FunctionA, the very first time, it crashes
with
an Illegal Instruction exception.

Does it crash before executing CallbackA() or after?

Assuming this guess is correct...

[...]
So, it's something about FunctionA's return parameter that's causing
this?
Doesn't make sense to me, but it sure looks that way.

....one possible explanation is that you've got an incorrect calling
convention. I don't know off the top of my head what requirement, if any,
..NET has with respect to calling convention. But given that at least one
significant calling convention difference will only show up in functions
that return values, that seems like a likely culprit to me. It certainly
could produce an illegal instruction exception, at least when trying to
return from FunctionA().

Off the top of my head, I'd guess .NET uses STDCALL, but you really ought
to just look it up and make sure.

I thought of this. To make matters a bit worse, this is actually a problem
on a Windows Mobile device that has a different convention, but I cannot see
that it's different with a parameter being passed. And, remember, by callback
functions both pass nothing (null) back. It's the function that's calling the
callback that has a different prototype (sorry if that's confusing).


What I started thinking about was that, for some reason, the garbage
colector was removing my delegate which is how I create the callback and then
pass it into my C DLL where the address looks OK. I don't know why this would
be and in my tests, it's only if I call the same callback from one of my
functions that this is a problem so I don't think that's right, either.


It's all very odd, indeed, and it does crash on the execution of the
CallbackA() line in my code (if I comment it out, all is fine... if I attach
a debugger it's when I step on that line that I see the exception thrown).

 

[bsnguy]

OK, I found the problem.

I was suspecting that the garbage collector was removing my callback for
some reason. It was. The problem was that I declared my callback's delegate
in the constructor for the class and not as a class variable. Hence, the GC
thought "hey, he's not in the class constructor any more so I can move/clean
up this no-longer-needed object." Moving it up to the class level fixed the
problem.

Ugh... nothing like posting to a list to make you find your stupid errors,
huh?

 

[Ben Voigt [C++ MVP]]

[...]
Ugh... nothing like posting to a list to make you find your stupid
errors,
huh?

Well, when you post to a newsgroup (not a "list"), and don't actually
include a concise-but-complete code example that reliably
demonstrates the problem, that's about the only way you will find
your errors, stupid or otherwise. No one else can point them out
when you don't share enough information for them to be seen.

By the way, if it's valid and expected for the callback to be used
only during the execution of the constructor, then you can use the
GC.KeepAlive() method instead of adding class-level storage for the
reference.

Conversion of delegates into native function pointers always causes lifetime
issues because the GC can't tell how long the native code holds the function
pointer. So you have to guarantee reachability yourself, and one of the
better ways to do this would be with GCHandle (since storing as a member
variable doesn't help if the instance becomes unreachable).

 

[bsnguy]

This doesn't work in my case as the callback is used in asynchronous threads
so there is really no where to put the GC.KeepAlive() call from what I can
tell, that will hold the reference.

Conversion of delegates into native function pointers always causes lifetime
issues because the GC can't tell how long the native code holds the function
pointer. So you have to guarantee reachability yourself, and one of the
better ways to do this would be with GCHandle (since storing as a member
variable doesn't help if the instance becomes unreachable).

Thanks for this tip. I might look into that for stability reasons.
Fortunately, the instance is required for the running of my application. That
is, if the instance became unreachable, the application would be
hung/dead/useless so I don't much care if my callback handle is gone.