marshaling a structure by value from managed to unmanaged

[Aston Martin]

Hi All,

********************** My Situation **********************
I am working on project that involves passing a structure to unmanaged
code from .Net world (well using C#). Perhaps an example will prove
useful.

structure MyStruct
{
// this is a complicated struct declaration in the sense
that it includes other structures
// (thankfully they are not variable sized) and also a
large array of char
SomeStruct1 member1
SomeStruct2 member2
char bigArray[7664];
};
void foo_get_struct_by_value( MyStruct valStruc );

in managed world I wrote all the required declaration for MyStruct,
SomeStruct1, SomeStruct2 and p-invoke signature of
foo_get_struct_by_value().

let this be the declaration in C#(appended Cs against each type, Cs
stands for well, C# )

structure CsMyStruct
{
CsSomeStruct1 member1
CsSomeStruct2 member2
char bigArray[7664];
};
void cs_foo_get_struct_by_value( MyStruct valStruc );

********************** My Problem **********************
if I try calling cs_foo_get_struct_by_value() MarshalDirectiveException
is being thrown with this message "Internal limitation: structure is
too complex or too large."

********************** My Solution (that didn't work, hence this
post) **********************
Hence I tried using unconventional means to achieve my goal. I cannot
modify the legacy code, it is important to achieve in managed code else
I will have to run around begging for permission to modify the legacy
code.
The idea is to present legacy code with a 'lump' of memory. The legacy
code will interpret it as a valid structure instance. Here is the code
illustrating what I am mean. For this I modified the p-invoke signature
of cs_foo_get_struct_by_value() to as below

cs_foo_get_struct_by_value( byte[] lumpOfMemory );

// allocate a structure and do something with it, fill it with data
etc.
CsMyStruct myStruct = new CsMyStruct();

// save the size, we will need it again
int structSize = Marshal.SizeOf( myStruct );

// allocate memory in unmanaged area
IntPtr ptr = Marshal.AllocHGlobal( structSize );
// get the address of managed structure
Marshal.StructureToPtr( myStruct, ptr, false );

// copy the managed structure contents in 'lump of memory'
byte[] lumpOfMemory = new byte[structSize];
Marshal.Copy( ptr, lumpOfMemory, 0, structSize );
Marshal.FreeHGlobal( ptr );

// let the unmanaged function do its job
cs_foo_get_struct_by_value( lumpOfMemory );

 

[Mattias Sjögren]

structure CsMyStruct

{
CsSomeStruct1 member1
CsSomeStruct2 member2
char bigArray[7664];
};

It would help if you could post valid C# code that actually compiles.

void cs_foo_get_struct_by_value( MyStruct valStruc );

Are you sure the struct should be passed by value and not by
reference. It's unusual to push such a large structure on the stack.



Mattias

 

[Aston Martin]

Hi

thanks for your interest, and sorry for my late reply, I have been busy
since late.
yes its unusual to push such large data to stack but as already told,
this is a legacy library, now I have made changes in legacy code to
take reference(pointer) to this structure and was able to marshal the
reference using IntPtr.

but am still interested in this problem. I will provide a working code
soon

regards,
Aston

structure CsMyStruct
{
CsSomeStruct1 member1
CsSomeStruct2 member2
char bigArray[7664];
};

It would help if you could post valid C# code that actually compiles.

void cs_foo_get_struct_by_value( MyStruct valStruc );

Are you sure the struct should be passed by value and not by
reference. It's unusual to push such a large structure on the stack.



Mattias

 

[Aston Martin]

below is the C# file

using System;
using System.Runtime.InteropServices;

namespace managed
{
[StructLayout(LayoutKind.Sequential, CharSet=CharSet.Unicode)]
struct CsSomeStruct1
{
internal int day;
internal int month;
internal int year;
}

[StructLayout(LayoutKind.Sequential, CharSet=CharSet.Unicode)]
struct CsMyStruct
{
internal CsSomeStruct1 member1;
internal CsSomeStruct1 member2;

[MarshalAs(UnmanagedType.ByValArray,
SizeConst=TestClass.BLOB_SIZE)]
internal byte[] bigArray;
}

/// <summary>
/// Summary description for Class1.
/// </summary>
class TestClass
{
public const int BLOB_SIZE = 75 * 1024;

/***************************** NOT-WORKING, BY VALUE
*****************************/
// Entry in unmanaged via value-taking-function.
// Calling this will cause MarshalDirectiveException to be
thrown
[DllImport("unmanaged.dll",
EntryPoint="foo_get_struct_by_value" ,CharSet=CharSet.Unicode)]
extern static void cs_foo_get_struct_by_value( CsMyStruct
valStruc );


/***************************** WORKING TRICK, BY REFERENCE
*****************************/
// Entry in unmanaged via reference-taking-function.
// This will work, but this is at the cost of introducing
// foo_get_struct_by_reference( MyStruct* pStruc ) from
unmanaged code.
[DllImport("unmanaged.dll",
EntryPoint="foo_get_struct_by_reference" ,CharSet=CharSet.Unicode)]
extern static void internal_cs_foo_get_struct_by_reference(
IntPtr pStruc );

// This will 'sheild' rest of code knowing about marshalling
and make
// them feel as if they are making a direct extern call.
static void cs_foo_get_struct_by_reference( CsMyStruct mystruc
)
{
IntPtr ptr = Marshal.AllocHGlobal( Marshal.SizeOf(
mystruc.GetType() ) );
Marshal.StructureToPtr( mystruc, ptr, false );
internal_cs_foo_get_struct_by_reference( ptr );
Marshal.FreeHGlobal( ptr );
}


/***************************** NOT-WORKING, BY BLOB
*****************************/
// Entry in unmanaged via value-taking-function, but we shall
present
// unmanaged code with a blob from managed code.
// This will not work, had this worked I would have continued
without
// changing legacy code.
[DllImport("unmanaged.dll",
EntryPoint="foo_get_struct_by_value" ,CharSet=CharSet.Unicode)]
extern static void internal_cs_foo_get_struct_by_blob( [In]
byte[] pBuffer );

// The idea is to present unamanged code with a blob from
managed memory
// which it shall treat as structure
// what i will do is take the pointer of managed structure and
copy the
// contents to an byte[] array and pass it to unmanaged code
static void cs_foo_get_struct_as_blob( CsMyStruct mystruc )
{
// allocate an array to pass to unmanaged code
int struct_native_size = Marshal.SizeOf( mystruc.GetType()
);
byte[] blob = new byte[struct_native_size];

// take the address of structure instance to pass
IntPtr ptr = Marshal.AllocHGlobal( struct_native_size );
Marshal.StructureToPtr( mystruc, ptr, false );

// copy the structure contents into that byte[] array
Marshal.Copy( ptr, blob, 0, struct_native_size );

// pass on to unmanaged code
internal_cs_foo_get_struct_by_blob( blob );

// free the memory to avoid memory-leaks
Marshal.FreeHGlobal( ptr );
}

/// <summary>
/// The main entry point for the a pplication.
/// </summary>
[STAThread]
static void Main(string[] args)
{
try
{
CsMyStruct mystruct = new CsMyStruct();
mystruct.bigArray = new byte[TestClass.BLOB_SIZE];
for( int i=0; i<BLOB_SIZE; i++ )
{
mystruct.bigArray = 10;
}

// the call below will throw a
MarshalDirectiveException
// cs_foo_get_struct_by_value( mystruct );

// this is passing a strucuture via blob, not working
// this will take to you unmanaged code but with
unexpected
// and potentially garbage data
cs_foo_get_struct_as_blob( mystruct );

// this call can take you to unamanaged code,
succesfully!
// cs_foo_get_struct_by_reference( mystruct );

}
// swallow all errors
catch( Exception error )
{
Console.WriteLine( error.Message );
}
}
}
}

and here is the unmanaged code

#include<tchar.h>

#define BLOB_SIZE 75 * 1024

struct SomeStruct1
{
int day;
int month;
int year;
};

struct MyStruct
{
// this is a complicated struct declaration in the sense
// that it includes other structures
// (thankfully they are not variable sized) and also a large array
of char
SomeStruct1 member1;
SomeStruct1 member2;
char bigArray[BLOB_SIZE];
};

// this function cannot be used since it wants structure by value
extern "C" __declspec(dllexport) void foo_get_struct_by_value( MyStruct
valStruc )
{
for( int i=0; i<BLOB_SIZE; i++ )
_tprintf( _T("%d"), valStruc.bigArray );
}

// this function cannot be used since it wants structure by reference
// suppose this code is part of a large legacy library and you somehow
// have to manage by calling the above value-taking function
extern "C" __declspec(dllexport) void foo_get_struct_by_reference(
MyStruct* pStruc )
{
foo_get_struct_by_value( *pStruc );
}

for this who are willing I can provide a working MS Studio solution
file with projects ofcourse

Hi

thanks for your interest, and sorry for my late reply, I have been busy
since late.
yes its unusual to push such large data to stack but as already told,
this is a legacy library, now I have made changes in legacy code to
take reference(pointer) to this structure and was able to marshal the
reference using IntPtr.

but am still interested in this problem. I will provide a working code
soon

regards,
Aston

structure CsMyStruct
{
CsSomeStruct1 member1
CsSomeStruct2 member2
char bigArray[7664];
};

It would help if you could post valid C# code that actually compiles.

Are you sure the struct should be passed by value and not by
reference. It's unusual to push such a large structure on the stack.

 

[Willy Denoyette [MVP]]

below is the C# file
....
for this who are willing I can provide a working MS Studio solution
file with projects ofcourse


The largest blob you can pass by value must be < 32KB, this is a CLR imposed interop
restriction (un-documented), anything larger must be passed by reference, or be passed
through a C++ managed wrapper.

Following is a working sample based on your code, but now using a fixed byte buffer (compile
with /unsafe). But as noted by Mattias, you should try avoid to pass such large blobs by
value between managed and unmanaged code.


using System;
using System.Runtime.InteropServices;

namespace managed
{
[StructLayout(LayoutKind.Sequential)]
struct CsSomeStruct1
{
internal int day;
internal int month;
internal int year;
}

[StructLayout(LayoutKind.Sequential)]
unsafe struct CsMyStruct
{
internal CsSomeStruct1 member1;
internal CsSomeStruct1 member2;
internal fixed byte bigArray[TestClass.BLOB_SIZE];
}
class TestClass
{
public const int BLOB_SIZE = 31 * 1024;

/***************************** NOT-WORKING, BY VALUE *****************************/
// Entry in unmanaged via value-taking-function.
// Calling this will cause MarshalDirectiveException to be thrown
[DllImport("passblobcpp.dll",EntryPoint="foo_get_struct_by_value" )]
extern static void cs_foo_get_struct_by_value( CsMyStruct valStruc );
/// <summary>
/// The main entry point for the a pplication.
/// </summary>
[STAThread]
static void Main(string[] args)
{
try
{
CsMyStruct mystruct = new CsMyStruct();
unsafe
{
for (int i = 0; i < BLOB_SIZE; i++)
{
mystruct.bigArray = 10;
}
}
// the call below will throw a MarshalDirectiveException
cs_foo_get_struct_by_value( mystruct );
}
// swallow all errors
catch( Exception error )
{
Console.WriteLine( error.Message );
}
}
}
}

 

[Aston Martin]

thanks to both of you Mattias as well as Willy for taking time to look
in this issue.

Willy,
what you told did work but only figures are different. In my case I
could carry upto 64K data through to unmanaged code. ( 63 * 1024 ). But
I did not understand the meaning of adding unsafe keyword, since it
worked for me without using it ?

regards,
Aston Martin

 

[Willy Denoyette [MVP]]

thanks to both of you Mattias as well as Willy for taking time to look
in this issue.

Willy,
what you told did work but only figures are different. In my case I
could carry upto 64K data through to unmanaged code. ( 63 * 1024 ). But
I did not understand the meaning of adding unsafe keyword, since it
worked for me without using it ?

Don't know exactly which unsafe keyword you are referring to, but I can compile the sample
posted without any of them.
Also, I can't get above 32 KB either, this all running on V2 of the framework with VS2005
SP1 (Beta1) installed, that means using version 8.00.50727.312 of the compiler.

Willy.