Native C Dll integration performance

[Frederic Van Belle]

Hi,

I am using in an C# application a native C DLL for image processing. The two
main functions we are using look like that :

int DLL_export _stdcall EncodeinProprietaryFormat
(
char
* i_image,
long
i_size,
char
** o_image,
long
* o_size,
);
int DLL_exporti stdcall Freeimage
(
char
** io_image
);

We manage to use those funtions with the code below :

……
[DllImport(
@".\encode.dll",
CharSet = CharSet.Ansi,
CallingConvention = CallingConvention.StdCall,
EntryPoint = @" EncodeinProprietaryFormat ")]
public static extern int EncodeinProprietaryFormat (
[In] byte[]
i_image,
[In] System.Int32 /*long*/
i_size,
[In,Out]ref IntPtr
o_image,
[In,Out] ref System.Int32
/*long*/ o_size
);
[DllImport(
@".\encode.dll",
CharSet = CharSet.Ansi,
CallingConvention = CallingConvention.StdCall,
EntryPoint = @" Freeimage†)]
public static extern int Freeimage (
[In, Out]ref IntPtr io_image
);
….
….

byte[] l_image = …..;
int l_imagesize = …. ;
IntPtr l_pointerencodedimage = IntPtr.Zero;
int l_encodedsize = 0;
byte[] l_encodedimage ;

unsafe
{
EncodeinProprietaryFormat (l_ image, imagesize, ref
l_pointerencodedimage, l_encodedsize) ;
l_ encodedimage = new byte[l_encodedsize];
for (int i = 0; i < l_encodedsize; i++)
{
l_encodedimage =
(byte)Marshal.PtrToStructure((IntPtr)((int) l_pointerencodedimage + i),
typeof(byte));
}
Freeimage (ref l_pointerencodedimage);

}

That code works but the for loop is time consuming and in the final
application we have to use that code for several million of pictures.

Is there a quickier solution ?

Best regards

Frederic Van Belle

 

[Frederic Van Belle]

Hi

Thank you for your quick answer.

.. Move the entire loop to C, :

Do you mean writing a small C dll that will exposed a single function that
deal with a fixed sized array of byte ?


Just use unsafe pointers directly in the
function signature :

in fact the unsafe brackets had been left due to a comprehension error from
my side
I have tried your suggestion without success :

For instance , if I rewrite

[DllImport(
@".\encode.dll",
CharSet = CharSet.Ansi,
CallingConvention = CallingConvention.StdCall,
EntryPoint = @" Freeimage†)]
public static extern int Freeimage (
[In, Out]ref IntPtr io_image
);

In

[DllImport(
@".\encode.dll",
CharSet = CharSet.Ansi,
CallingConvention = CallingConvention.StdCall,
EntryPoint = @" Freeimage†)]
public static extern int Freeimage (
[In, Out]byte** io_image
);

I get an error : Pointers and fixed size buffers may only be used in an
unsafe context

And if I put

Unsafe
{
[DllImport(
@".\encode.dll",
CharSet = CharSet.Ansi,
CallingConvention = CallingConvention.StdCall,
EntryPoint = @" Freeimage†)]
public static extern int Freeimage (
[In, Out]byte** io_image
);

}

I get a lot of compilation error as I think unsafe brackets can not be used
in declaration

Best regards

 

[Pavel Minaev]

Do you mean writing a small C dll  that will exposed a single function that
deal with a fixed sized array of byte ?

You're saying that you have a long loop that calls the same set of
functions from a C DLL. My suggestion is to move the entire loop to
that same DLL, and export it as a single function with all the
necessary parameters, so that you only need to make a single call.

I have tried your suggestion without success :

For instance , if I rewrite

        [DllImport(
            @".\encode.dll",
            CharSet = CharSet.Ansi,
            CallingConvention = CallingConvention.StdCall,
            EntryPoint = @" Freeimage” )]
        public static extern int  Freeimage (
                                                [In, Out]ref IntPtr io_image
                                                );

In

[DllImport(
            @".\encode.dll",
            CharSet = CharSet.Ansi,
            CallingConvention = CallingConvention.StdCall,
            EntryPoint = @" Freeimage” )]
        public static extern int  Freeimage (
                                                [In, Out]byte** io_image
                                                );

I get an error :  Pointers and fixed size buffers may only be used in an
unsafe context

You should mark the method itself as unsafe with the "unsafe"
_modifier_:

public static extern unsafe int FreeImage(byte** io_image);

or the entire class, if you only have P/Invoke declarations in it:

unsafe class MyDllExports { ... }

The unsafe _statement_ ("unsafe { ... }") is only valid where
statements in general are valid - i.e., in method bodies.

By the way, note that [In,Out] is not needed if you work with pointers
directly (either IntPtr or proper pointers), since you deal with all
marshalling yourself there. It's only needed for ref and arrays, so
that P/Invoke knows whether it needs to copy the data back or not.

 

[Frederic Van Belle]

Hi

You're saying that you have a long loop that calls the same set of
functions from a C DLL. My suggestion is to move the entire loop to
that same DLL, and export it as a single function with all the
necessary parameters, so that you only need to make a single call.

No in fact in the code provided the loop do recopy (like memcpy) the data
from the unmanaged memory area to the managed memory area

for (int i = 0; i < l_encodedsize; i++)
{
l_encodedimage =
(byte)Marshal.PtrToStructure((IntPtr)((int) l_pointerencodedimage + i),
typeof(byte));
}

Thank you.

 

[Pavel Minaev]

Hi

You're saying that you have a long loop that calls the same set of
functions from a C DLL. My suggestion is to move the entire loop to
that same DLL, and export it as a single function with all the
necessary parameters, so that you only need to make a single call.

No in fact in the code provided the loop do recopy (like memcpy) the data
from the unmanaged memory area to the managed memory area

  for (int i = 0; i < l_encodedsize; i++)
                    {
                        l_encodedimage =
(byte)Marshal.PtrToStructure((IntPtr)((int) l_pointerencodedimage + i),
typeof(byte));
                    }

OMG! I've missed what's actually going on inside that loop, and of
course that thing is going to be a performance killer.

Have a look at Marshal.Copy. It does precisely what you need - takes
an IntPtr and length, and copies the data pointed to by that IntPtr
into an array, efficiently.

However, it would be even better to try to avoid the copy altogether.
One way would be to deal with the returned data directly, without
copying it into an array (and that's where declaring it as byte*
rather than IntPtr will help), and postpone calling FreeImage until
you are done with it. However, it's pretty easy to forget to free on
some code paths, and leak memory, so you'll need to be careful with
that.

 

[Frederic Van Belle]

Hi Pavel,

Thanks for your answers it was really helpful

Kind regards!

Hi

You're saying that you have a long loop that calls the same set of
functions from a C DLL. My suggestion is to move the entire loop to
that same DLL, and export it as a single function with all the
necessary parameters, so that you only need to make a single call.

No in fact in the code provided the loop do recopy (like memcpy) the data
from the unmanaged memory area to the managed memory area

for (int i = 0; i < l_encodedsize; i++)
{
l_encodedimage =
(byte)Marshal.PtrToStructure((IntPtr)((int) l_pointerencodedimage + i),
typeof(byte));
}

OMG! I've missed what's actually going on inside that loop, and of
course that thing is going to be a performance killer.

Have a look at Marshal.Copy. It does precisely what you need - takes
an IntPtr and length, and copies the data pointed to by that IntPtr
into an array, efficiently.

However, it would be even better to try to avoid the copy altogether.
One way would be to deal with the returned data directly, without
copying it into an array (and that's where declaring it as byte*
rather than IntPtr will help), and postpone calling FreeImage until
you are done with it. However, it's pretty easy to forget to free on
some code paths, and leak memory, so you'll need to be careful with
that.