BUG: CallingConvention not followed

[Fernando Cacciola]

I have the following C# declaration:

[DllImport("mylib.dll", CallingConvention = CallingConvention.StdCall)]
extern static bool SomeFunc( IntPtr a
,int [] b
,byte [] c
,PointF[] d
);

which calls into some unmanaged C++ function.

I'm using VS.NET 2003

The dissasembly output shows:

000000b9 push dword ptr [ebp-1Ch]
000000bc push dword ptr [ebp-14h]
000000bf mov edx,dword ptr [ebp-18h]
000000c2 mov ecx,dword ptr [ebp-4]
000000c5 call dword ptr ds:[09092E40h]

But this is wrong!!!!
only the last two parameters are being pushed onto the stack
the other two are passed in the edx and ecx registers.

naturally, it crashes, because the C++
functions expected the four arguments in the
stack.

I noticed that this has been reported here before:

http://tinyurl.com/4mgak

TIA

Fernando Cacciola

 

[Mattias Sjögren]

Fernando,

The dissasembly output shows:

000000b9 push dword ptr [ebp-1Ch]
000000bc push dword ptr [ebp-14h]
000000bf mov edx,dword ptr [ebp-18h]
000000c2 mov ecx,dword ptr [ebp-4]
000000c5 call dword ptr ds:[09092E40h]

But this is wrong!!!!
only the last two parameters are being pushed onto the stack
the other two are passed in the edx and ecx registers.

Keep in mind that all P/Invoke calls go through some thunk code that
handles marshalling and stuff. So the stack that's set up here isn't
necessarily the same as what it will look like when the actual native
function is called. The CLR tends to use the __fastcall calling
convention internally, so the assembler looks correct to me.

If calls to __stdcall functions were this fundamentally broken, I
think it would have been found a long time ago, because they are used
everywhere. Most of the Win32 API, on which the BCL depends heavily,
use __stdcall.

naturally, it crashes, because the C++

Perhaps you should look for other reasons for the crash.



Mattias

 

[Fernando Cacciola]

Fernando,

The dissasembly output shows:

000000b9 push dword ptr [ebp-1Ch]
000000bc push dword ptr [ebp-14h]
000000bf mov edx,dword ptr [ebp-18h]
000000c2 mov ecx,dword ptr [ebp-4]
000000c5 call dword ptr ds:[09092E40h]

But this is wrong!!!!
only the last two parameters are being pushed onto the stack
the other two are passed in the edx and ecx registers.

Keep in mind that all P/Invoke calls go through some thunk code that
handles marshalling and stuff. So the stack that's set up here isn't
necessarily the same as what it will look like when the actual native
function is called. The CLR tends to use the __fastcall calling
convention internally, so the assembler looks correct to me.

Indeed...
I watched the stack memory contents, stepped into the "call" assembler
instruction, and there, right at the "push bp" of the C++ function prolog,
the stack suddenly appeared with a lot more stuff than what it had "just 1
instruction before".
Clearly, the Visual Studio dissasembly window is shortcutting and hiding
code... the relevant parts of the code in this case, unfortunately.

If calls to __stdcall functions were this fundamentally broken, I
think it would have been found a long time ago, because they are used
everywhere. Most of the Win32 API, on which the BCL depends heavily,
use __stdcall.

Right... I thought it had to do with this very particular function, but then
I tried with a simple test function and saw the same pattern.
Unfortunately, I can't see the "real code"...
Is there any "lower" level debugger I can use? or even better, any
documentation on the _details_ of P/Invoke?
TIA

Perhaps you should look for other reasons for the crash.

Well, for whatever reason it worked when I added [Out] to the parameter:
PointF[] d

In the C++ side this is: PointF const* d

Thanks

Fernando Cacciola
SciSoft

 

[Mattias Sjögren]

Fernando,

Unfortunately, I can't see the "real code"...
Is there any "lower" level debugger I can use?

VS can do this, it just takes some extra work. What I usually do when
I want to step through the CLR code is to call DebugBreak() to attach
to the debugger, and make sure I select to only use native debugging.



Mattias

 

[Fernando Cacciola]

Fernando,


VS can do this, it just takes some extra work. What I usually do when
I want to step through the CLR code is to call DebugBreak() to attach
to the debugger, and make sure I select to only use native debugging.

Oh, thanks!

Fernando Cacciola
SciSoft