Problem: Calling an unmanaged function returning structure by value

[mczard]

The MSDN documentation explains how to pass structures from C# to C
functions, but I can't find any help on how to return a structure by
value from C to C#. I don't want to return a pointer to the structure,
because it would require additional memory management.

I tried this:

------------------------------------------------------------------------
C:

struct Point
{
int x;
int y;
};

extern "C"
__declspec(dllexport)
Point proc3()
{
Point p;
p.x = 3;
p.y = 4;
return p;
}

------------------------------------------------------------------------
C#:

delegate Point Proc3();
....
IntPtr lib = LoadLibrary("../../../debug/CeeLibrary.dll");
IntPtr proc3 = GetProcAddress(lib, "proc3");
Proc3 del3 = (Proc3)Marshal.GetDelegateForFunctionPointer(proc3,
typeof(Proc3));
Point p3 = del3();

------------------------------------------------------------------------

and it does work, but it crashes if I add an integer parameter:

------------------------------------------------------------------------
C:

extern "C"
__declspec(dllexport)
Point proc4(int x)
{
Point p;
p.x = x;
p.y = 4;
return p;
}

------------------------------------------------------------------------
C#:

delegate Point Proc4(int x);
....
IntPtr lib = LoadLibrary("../../../debug/CeeLibrary.dll");
IntPtr proc4 = GetProcAddress(lib, "proc4");
Proc4 del4 = (Proc4)Marshal.GetDelegateForFunctionPointer(proc4,
typeof(Proc4));
Point p4 = del4(10);

 

[Willy Denoyette [MVP]]

The MSDN documentation explains how to pass structures from C# to C
functions, but I can't find any help on how to return a structure by
value from C to C#. I don't want to return a pointer to the structure,
because it would require additional memory management.

I tried this:

------------------------------------------------------------------------
C:

struct Point
{
int x;
int y;
};

extern "C"
__declspec(dllexport)
Point proc3()
{
Point p;
p.x = 3;
p.y = 4;
return p;
}

------------------------------------------------------------------------
C#:

delegate Point Proc3();
...
IntPtr lib = LoadLibrary("../../../debug/CeeLibrary.dll");
IntPtr proc3 = GetProcAddress(lib, "proc3");
Proc3 del3 = (Proc3)Marshal.GetDelegateForFunctionPointer(proc3,
typeof(Proc3));
Point p3 = del3();

------------------------------------------------------------------------

and it does work, but it crashes if I add an integer parameter:

------------------------------------------------------------------------
C:

extern "C"
__declspec(dllexport)
Point proc4(int x)
{
Point p;
p.x = x;
p.y = 4;
return p;
}

------------------------------------------------------------------------
C#:

delegate Point Proc4(int x);
...
IntPtr lib = LoadLibrary("../../../debug/CeeLibrary.dll");
IntPtr proc4 = GetProcAddress(lib, "proc4");
Proc4 del4 = (Proc4)Marshal.GetDelegateForFunctionPointer(proc4,
typeof(Proc4));
Point p4 = del4(10);

Don't know why you are using LoadLibrary/GetProcAddress instead of a simple
DllImport declaration here.
Anyway, you are dealing with a stack imbalance due to a calling convention
mismatch, you are calling a __cdecl function however the default calling
convention used by interop is __stdcall you need to specify the Cdecl
calling convention when calling these functions.

[DllImport("lib",CallingConvention = CallingConvention.Cdecl)

or, as in your case:

[UnmanagedFunctionPointerAttribute( CallingConvention.Cdecl)]
delegate Point Proc3();

Willy.

 

[mczard]

Don't know why you are using LoadLibrary/GetProcAddress instead of a simple
DllImport declaration here.

I am writing an interface for plug-ins and the application will not
know (at compile time) what plug-ins are installed.

Anyway, you are dealing with a stack imbalance due to a calling convention
mismatch, you are calling a __cdecl function however the default calling
convention used by interop is __stdcall you need to specify the Cdecl
calling convention when calling these functions.

[DllImport("lib",CallingConvention = CallingConvention.Cdecl)

This works, but the static (dll loading) solution works even without
the explicit calling conversion. Even more - it works with both Cdecl
and StdCall!

or, as in your case:

[UnmanagedFunctionPointerAttribute( CallingConvention.Cdecl)]
delegate Point Proc3();

And this unfortunatelly still doesn't solve the problem - the
application
still crashes when working with:

[UnmanagedFunctionPointerAttribute( CallingConvention.Cdecl)]
delegate Point Proc4(int y);

It's strange, but the problem does not appear if there are no
parameters
in the function.

Here is the complete minimal VS solution (a C project + a C# project):
www.future-processing.com/~mczardybon/InteropProblemMini.zip

Could you, please, have a look at it?

 

[mczard]

After 2 days I have found a weird work-around:

delegate Int64 Proc4(int x);
....
Int64 n = proc4_dynamic(11);
Point p;
p.x = (int)(n % 4294967296L);
p.y = (int)(n / 4294967296L);

(the C library remain the same).

I.e. explointing the fact that Point structure has 64 bits, I force C#
to
mashall it as an Int64 and then I can extract the actual point
coordinates.

But this is strange and not scalable. There should be a better way.
It seems to me, as it is an error in the libraries. I use VS2005...

 

[Willy Denoyette [MVP]]

I am writing an interface for plug-ins and the application will not
know (at compile time) what plug-ins are installed.

Well, a simple solution for this is installing the plug-ins in a fixed
location or a few locations, and add this location to the path environment
variable.

Anyway, you are dealing with a stack imbalance due to a calling
convention
mismatch, you are calling a __cdecl function however the default calling
convention used by interop is __stdcall you need to specify the Cdecl
calling convention when calling these functions.

[DllImport("lib",CallingConvention = CallingConvention.Cdecl)

This works, but the static (dll loading) solution works even without
the explicit calling conversion. Even more - it works with both Cdecl
and StdCall!

That's normal, from the import signature, the PInvoke layer has sufficient
knowledge of the real calling convention required in order to adjust the
stack imbalance at run-time. However this is not true when explicit
loading/calling is used, here you have to declare your function with correct
calling convention, you should take the habit to specify the correct calling
convention when using implicit loading.

or, as in your case:

[UnmanagedFunctionPointerAttribute( CallingConvention.Cdecl)]
delegate Point Proc3();

And this unfortunatelly still doesn't solve the problem - the
application
still crashes when working with:

[UnmanagedFunctionPointerAttribute( CallingConvention.Cdecl)]
delegate Point Proc4(int y);

It's strange, but the problem does not appear if there are no
parameters
in the function.

This is because the Interop layer cannot handle this correctly when using
explicit loading/calling.
So , passing a struct is not possible as *return* of a function call, one
option to solve this is to return a pointer to a callee allocated structure,
the callee needs to marshal the contents to the struct after the call
returns. The problem with this option, is that the callee needs to provide a
way for the caller to free the memory allocated. Another option 2) is to
pass the structure ref to a caller as argument.

1) Callee allocated struct
//C
extern "C"
__declspec(dllexport) Point* __cdecl proc5(int v)
{
Point *p = new Point();
p->x = 3;
p->y = 4;
return p;
}

//C#
[UnmanagedFunctionPointerAttribute( CallingConvention.Cdecl)]
public delegate IntPtr Proc3(int i);
....
int v = 5;
Proc3 del3 =
(Proc3)Marshal.GetDelegateForFunctionPointer(proc3, typeof(Proc3));
IntPtr ptr = del3(v);
Point p = (Point)Marshal.PtrToStructure(ptr, typeof(Point));
// NEED to implement/call a function that frees the callees
allocated struct!!
...
2) Caller allocated struct.

//C
extern "C"
__declspec(dllexport) void __cdecl proc3(Point *p, int v)
{
p->x = 3;
p->y = 4;
return p;
}
C#
[UnmanagedFunctionPointerAttribute( CallingConvention.Cdecl)]
public delegate void Proc3(ref Point pPoint, int i);
...
Point p2 = new Point();
Proc3 del3 = (Proc3)Marshal.GetDelegateForFunctionPointer(proc3,
typeof(Proc3));
del3(ref p2, 7);
....


Wolly.

 

[Willy Denoyette [MVP]]

After 2 days I have found a weird work-around:

delegate Int64 Proc4(int x);
...
Int64 n = proc4_dynamic(11);
Point p;
p.x = (int)(n % 4294967296L);
p.y = (int)(n / 4294967296L);

(the C library remain the same).

I.e. explointing the fact that Point structure has 64 bits, I force C#
to
mashall it as an Int64 and then I can extract the actual point
coordinates.

This is bad as it only works for this specific structure and you are messing
with the interop marshaler, no-one will guaranteed that it will work with a
newer version of the framework.

But this is strange and not scalable. There should be a better way.
It seems to me, as it is an error in the libraries. I use VS2005...

What do you mean with an error in the libraries?
The interop layer cannot deal with structure returns in case of *explicit*
binding as I said in my previous reply. The interop marshaler can only
handle this when implicit binding is used (DllImport).
More, the interop marshaler cannot deal with structures containing non
blittable types (irrespective the binding used), in such cases you need to
pass the structure by address (an IntPtr) and do the marshaling yourself.

Willy.

 

[mczard]

And this unfortunatelly still doesn't solve the problem - the

application still crashes when working with:
[UnmanagedFunctionPointerAttribute( CallingConvention.Cdecl)]
delegate Point Proc4(int y);

This is because the Interop layer cannot handle this correctly when using
explicit loading/calling.

Thank you for your help. It seems that I have to surrender and change
the signature of my function.

However, I still cannot understand why Interop is able to do something
with DllImport, but is not able to do the same thing with LoadLibrary/
GetDelegateForFunctionPointer... Where is the point, where the
information
is missing in the second case?

 

[Willy Denoyette [MVP]]

And this unfortunatelly still doesn't solve the problem - the
application still crashes when working with:
[UnmanagedFunctionPointerAttribute( CallingConvention.Cdecl)]
delegate Point Proc4(int y);

This is because the Interop layer cannot handle this correctly when using
explicit loading/calling.

Thank you for your help. It seems that I have to surrender and change
the signature of my function.

However, I still cannot understand why Interop is able to do something
with DllImport, but is not able to do the same thing with LoadLibrary/
GetDelegateForFunctionPointer... Where is the point, where the
information
is missing in the second case?

The Interop layer can to the right thing in case of implicit binding
(supposing that the C# declaration matches the C definition), because this
information is made available at compile time from the DllImport attribute,
some of this information is not available when using explicit binding at
compile time, the current interop layer does not handle this corner case at
run-time. The result of this is that the JIT compiler can generate a thunk
that handles "non-primitives" returns correctly (to a certain extend), in
case of explicit binding, the thunk is a generic thunk that cannot handle
the fact that you are passing a hidden argument in the call (the address of
Point at the caller).
If you need to know more about what's happening at the caller and at the
callee and at the interop layer, just run a simple test case in a native
debugger.

Willy.

 

[Willy Denoyette [MVP]]

And this unfortunatelly still doesn't solve the problem - the
application still crashes when working with:
[UnmanagedFunctionPointerAttribute( CallingConvention.Cdecl)]
delegate Point Proc4(int y);

This is because the Interop layer cannot handle this correctly when
using
explicit loading/calling.

Thank you for your help. It seems that I have to surrender and change
the signature of my function.

However, I still cannot understand why Interop is able to do something
with DllImport, but is not able to do the same thing with LoadLibrary/
GetDelegateForFunctionPointer... Where is the point, where the
information
is missing in the second case?

The Interop layer can to the right thing in case of implicit binding
(supposing that the C# declaration matches the C definition), because this
information is made available at compile time from the DllImport
attribute, some of this information is not available when using explicit
binding at compile time, the current interop layer does not handle this
corner case at run-time. The result of this is that the JIT compiler can
generate a thunk that handles "non-primitives" returns correctly (to a
certain extend), in case of explicit binding, the thunk is a generic thunk
that cannot handle the fact that you are passing a hidden argument in the
call (the address of Point at the caller).
If you need to know more about what's happening at the caller and at the
callee and at the interop layer, just run a simple test case in a native
debugger.

Willy.

One thing you could do is remove the extern "C" from the function
declaration and use the "decorated" name of the function to call, this
decorated name can be found by running dumpbin /exports on the dll. The name
should look something like : ?proc3@@YA?AUPoint@@H@Z.
This should give the necessary information to the Interop layer to handle
the hidden argument.

Willy.

 

[Willy Denoyette [MVP]]

And this unfortunatelly still doesn't solve the problem - the
application still crashes when working with:
[UnmanagedFunctionPointerAttribute( CallingConvention.Cdecl)]
delegate Point Proc4(int y);

This is because the Interop layer cannot handle this correctly when
using
explicit loading/calling.

Thank you for your help. It seems that I have to surrender and change
the signature of my function.

However, I still cannot understand why Interop is able to do something
with DllImport, but is not able to do the same thing with LoadLibrary/
GetDelegateForFunctionPointer... Where is the point, where the
information
is missing in the second case?

The Interop layer can to the right thing in case of implicit binding
(supposing that the C# declaration matches the C definition), because
this information is made available at compile time from the DllImport
attribute, some of this information is not available when using explicit
binding at compile time, the current interop layer does not handle this
corner case at run-time. The result of this is that the JIT compiler can
generate a thunk that handles "non-primitives" returns correctly (to a
certain extend), in case of explicit binding, the thunk is a generic
thunk that cannot handle the fact that you are passing a hidden argument
in the call (the address of Point at the caller).
If you need to know more about what's happening at the caller and at the
callee and at the interop layer, just run a simple test case in a native
debugger.

Willy.

One thing you could do is remove the extern "C" from the function
declaration and use the "decorated" name of the function to call, this
decorated name can be found by running dumpbin /exports on the dll. The
name should look something like : ?proc3@@YA?AUPoint@@H@Z.
This should give the necessary information to the Interop layer to handle
the hidden argument.

Willy.

Ok, forget about the above, a quick test of the above throws the same
exception (Access Violation). It's clear that Interop wasn't designed to
handle this scenario other than using DllImport.

Willy.