Mixing a native DLL with a managed DLL on CF - solved

[Robert Simpson]

I've been searching high and low for the ability to do this on the Compact
Framework. In a nutshell, I have a 100% native DLL and I want to fold a
100% pure .NET assembly into it (the wrapper for it). At first I thought
this would be pretty simple, since it is so easy to do on the full
framework. Boy was I wrong!

Originally I tried the old link a netmodule thing (adding a netmodule to the
linker's INPUT parameter), but the CF linker either blew up or gave me an
error. Works great on the full framework.

I tried lots and lots of combinations, but nothing worked. I hand-edited
netmodules, poked the linker in uncomfortable ways, etc. No joy.

Sheer determination ended up winning the battle. If I can't make the linker
do it, I'll do it myself!

This proved rather daunting. I started with Matt Pietrek's documentation on
the PE file format, and dug in all the source I could find on reading .NET
metadata. I finally settled down with the ECMA spec and coded up some
classes to read & modify metadata and tables.

In order to merge a .NET module, a CLR header has to be inserted into the
native DLL, and all the .NET metadata and IL as well. Then, since you're
not inserting it into the same relative places as they used to live, all the
RVA's have to be fixed up. This proved challenging.

Fixing up the metadata RVA's was easy. Having to write all the table code
to dig into the .NET metadata tables to fetch the MethodDef's and FieldRVA's
was a pain in the butt!

In the end, I got what I wanted. Source available here:
http://sqlite.phxsoftware.com/files/mergebin.zip

And that brings me to the reason for the post ...

The source isn't well documented yet -- I plan on writing up an article or
something for Codeguru in the near future. However, the code has some
caveats I'm hoping someone will know how to resolve ...

Caveats:
1. The native DLL must have an empty section in it large enough to hold the
..NET header, metadata and IL code. To create such a section, I used this:
#pragma data_seg(\".clr\")
#pragma comment(linker, "/SECTION:.clr,ER")
char __ph[92316] = {0}; // 92316 is the number of bytes to reserve
#pragma data_seg()

2. The native DLL must have an exported function in it that calls
CorDllMain() in MSCOREE.DLL. The function name must have "CORDLLMAIN" in it
(search is case-insensitive) and must have the same calling conventions as
DllMain. The reason I require this is because I have to remap the DLL
EntryPoint to call this function, and set the entrypoint in the .NET header
to be the previous DLL entrypoint. The reason it must be exported is
because I don't know how to figure out the RVA of the function without it
being in the export table. Here's my stub:

extern BOOL WINAPI _CorDllMain(HINSTANCE, DWORD, LPVOID);
__declspec(dllexport) BOOL WINAPI _MyCorDllMain(HINSTANCE hInst,
DWORD dwReason,
LPVOID pvReserved)
{
return _CorDllMain(hInst, dwReason, pvReserved);
}

3. The code is largely untested -- I tested it for my own usage, but your
mileage may vary. Oh, if the .NET assembly is signed, you'll have to
re-sign it afterwards.

So that leads to the questions:

Is there a more graceful way to do this? I'd much rather expand the size of
the executable than have to hardcode a reserved space for the assembly --
but at the same time I don't want to write out data into a non-existent
section of the file. Should I care? Will PEVerify complain? It already
complains about my non-standard stub.

Is there a way to set the entrypoint of the DLL directly to CorDllMain
rather than force you to write a stub that calls it?

Robert

 

[Alex Yakhnin [MVP]]

Robert,

I am sorry, but I don't understand the purpose. What's the end goal you're
trying to achieve?

--
Alex Yakhnin, .NET CF MVP
www.intelliprog.com | www.opennetcf.org

I've been searching high and low for the ability to do this on the Compact
Framework. In a nutshell, I have a 100% native DLL and I want to fold a
100% pure .NET assembly into it (the wrapper for it). At first I thought
this would be pretty simple, since it is so easy to do on the full
framework. Boy was I wrong!

Originally I tried the old link a netmodule thing (adding a netmodule to
the linker's INPUT parameter), but the CF linker either blew up or gave me
an error. Works great on the full framework.

I tried lots and lots of combinations, but nothing worked. I hand-edited
netmodules, poked the linker in uncomfortable ways, etc. No joy.

Sheer determination ended up winning the battle. If I can't make the
linker do it, I'll do it myself!

This proved rather daunting. I started with Matt Pietrek's documentation
on the PE file format, and dug in all the source I could find on reading
.NET metadata. I finally settled down with the ECMA spec and coded up
some classes to read & modify metadata and tables.

In order to merge a .NET module, a CLR header has to be inserted into the
native DLL, and all the .NET metadata and IL as well. Then, since you're
not inserting it into the same relative places as they used to live, all
the RVA's have to be fixed up. This proved challenging.

Fixing up the metadata RVA's was easy. Having to write all the table code
to dig into the .NET metadata tables to fetch the MethodDef's and
FieldRVA's was a pain in the butt!

In the end, I got what I wanted. Source available here:
http://sqlite.phxsoftware.com/files/mergebin.zip

And that brings me to the reason for the post ...

The source isn't well documented yet -- I plan on writing up an article or
something for Codeguru in the near future. However, the code has some
caveats I'm hoping someone will know how to resolve ...

Caveats:
1. The native DLL must have an empty section in it large enough to hold
the .NET header, metadata and IL code. To create such a section, I used
this:
#pragma data_seg(\".clr\")
#pragma comment(linker, "/SECTION:.clr,ER")
char __ph[92316] = {0}; // 92316 is the number of bytes to reserve
#pragma data_seg()

2. The native DLL must have an exported function in it that calls
CorDllMain() in MSCOREE.DLL. The function name must have "CORDLLMAIN" in
it (search is case-insensitive) and must have the same calling conventions
as DllMain. The reason I require this is because I have to remap the DLL
EntryPoint to call this function, and set the entrypoint in the .NET
header to be the previous DLL entrypoint. The reason it must be exported
is because I don't know how to figure out the RVA of the function without
it being in the export table. Here's my stub:

extern BOOL WINAPI _CorDllMain(HINSTANCE, DWORD, LPVOID);
__declspec(dllexport) BOOL WINAPI _MyCorDllMain(HINSTANCE hInst,
DWORD dwReason,
LPVOID pvReserved)
{
return _CorDllMain(hInst, dwReason, pvReserved);
}

3. The code is largely untested -- I tested it for my own usage, but your
mileage may vary. Oh, if the .NET assembly is signed, you'll have to
re-sign it afterwards.

So that leads to the questions:

Is there a more graceful way to do this? I'd much rather expand the size
of the executable than have to hardcode a reserved space for the
assembly -- but at the same time I don't want to write out data into a
non-existent section of the file. Should I care? Will PEVerify complain?
It already complains about my non-standard stub.

Is there a way to set the entrypoint of the DLL directly to CorDllMain
rather than force you to write a stub that calls it?

Robert

 

[Robert Simpson]

Really? I thought I was pretty clear.

I have a native DLL, and I have a .NET wrapper for it. I want to combine
them into a single DLL so that the DLL can be used both natively by native
components, and the wrapper is used by managed components. In either
environment, the same DLL is used, and the wrapper P/Invokes into itself.

On the full framework, this is easy. You merely compile your .NET wrapper
into a .netmodule and add a reference to the .netmodule as an input to the
linker in the native DLL's project settings. You can't do this on the
Compact Framework though. The linker will throw all sorts of errors if you
try.

Robert

Robert,

I am sorry, but I don't understand the purpose. What's the end goal you're
trying to achieve?

--
Alex Yakhnin, .NET CF MVP
www.intelliprog.com | www.opennetcf.org

I've been searching high and low for the ability to do this on the
Compact Framework. In a nutshell, I have a 100% native DLL and I want to
fold a 100% pure .NET assembly into it (the wrapper for it). At first I
thought this would be pretty simple, since it is so easy to do on the
full framework. Boy was I wrong!

Originally I tried the old link a netmodule thing (adding a netmodule to
the linker's INPUT parameter), but the CF linker either blew up or gave
me an error. Works great on the full framework.

I tried lots and lots of combinations, but nothing worked. I hand-edited
netmodules, poked the linker in uncomfortable ways, etc. No joy.

Sheer determination ended up winning the battle. If I can't make the
linker do it, I'll do it myself!

This proved rather daunting. I started with Matt Pietrek's documentation
on the PE file format, and dug in all the source I could find on reading
.NET metadata. I finally settled down with the ECMA spec and coded up
some classes to read & modify metadata and tables.

In order to merge a .NET module, a CLR header has to be inserted into the
native DLL, and all the .NET metadata and IL as well. Then, since you're
not inserting it into the same relative places as they used to live, all
the RVA's have to be fixed up. This proved challenging.

Fixing up the metadata RVA's was easy. Having to write all the table
code to dig into the .NET metadata tables to fetch the MethodDef's and
FieldRVA's was a pain in the butt!

In the end, I got what I wanted. Source available here:
http://sqlite.phxsoftware.com/files/mergebin.zip

And that brings me to the reason for the post ...

The source isn't well documented yet -- I plan on writing up an article
or something for Codeguru in the near future. However, the code has some
caveats I'm hoping someone will know how to resolve ...

Caveats:
1. The native DLL must have an empty section in it large enough to hold
the .NET header, metadata and IL code. To create such a section, I used
this:
#pragma data_seg(\".clr\")
#pragma comment(linker, "/SECTION:.clr,ER")
char __ph[92316] = {0}; // 92316 is the number of bytes to reserve
#pragma data_seg()

2. The native DLL must have an exported function in it that calls
CorDllMain() in MSCOREE.DLL. The function name must have "CORDLLMAIN" in
it (search is case-insensitive) and must have the same calling
conventions as DllMain. The reason I require this is because I have to
remap the DLL EntryPoint to call this function, and set the entrypoint in
the .NET header to be the previous DLL entrypoint. The reason it must be
exported is because I don't know how to figure out the RVA of the
function without it being in the export table. Here's my stub:

extern BOOL WINAPI _CorDllMain(HINSTANCE, DWORD, LPVOID);
__declspec(dllexport) BOOL WINAPI _MyCorDllMain(HINSTANCE hInst,
DWORD dwReason,
LPVOID pvReserved)
{
return _CorDllMain(hInst, dwReason, pvReserved);
}

3. The code is largely untested -- I tested it for my own usage, but
your mileage may vary. Oh, if the .NET assembly is signed, you'll have
to re-sign it afterwards.

So that leads to the questions:

Is there a more graceful way to do this? I'd much rather expand the size
of the executable than have to hardcode a reserved space for the
assembly -- but at the same time I don't want to write out data into a
non-existent section of the file. Should I care? Will PEVerify
complain? It already complains about my non-standard stub.

Is there a way to set the entrypoint of the DLL directly to CorDllMain
rather than force you to write a stub that calls it?

Robert

 

[Guest]

OK, so what is the advantage then? What are you winning in the end aside from
being able to distribute one DLL and adding more complexity to the future
upgrades and maintainence.

--
Alex Yakhnin, .NET CF MVP
www.intelliprog.com | www.opennetcf.org

Really? I thought I was pretty clear.

I have a native DLL, and I have a .NET wrapper for it. I want to combine
them into a single DLL so that the DLL can be used both natively by native
components, and the wrapper is used by managed components. In either
environment, the same DLL is used, and the wrapper P/Invokes into itself.

On the full framework, this is easy. You merely compile your .NET wrapper
into a .netmodule and add a reference to the .netmodule as an input to the
linker in the native DLL's project settings. You can't do this on the
Compact Framework though. The linker will throw all sorts of errors if you
try.

Robert

Robert,

I am sorry, but I don't understand the purpose. What's the end goal you're
trying to achieve?

--
Alex Yakhnin, .NET CF MVP
www.intelliprog.com | www.opennetcf.org

I've been searching high and low for the ability to do this on the
Compact Framework. In a nutshell, I have a 100% native DLL and I want to
fold a 100% pure .NET assembly into it (the wrapper for it). At first I
thought this would be pretty simple, since it is so easy to do on the
full framework. Boy was I wrong!

Originally I tried the old link a netmodule thing (adding a netmodule to
the linker's INPUT parameter), but the CF linker either blew up or gave
me an error. Works great on the full framework.

I tried lots and lots of combinations, but nothing worked. I hand-edited
netmodules, poked the linker in uncomfortable ways, etc. No joy.

Sheer determination ended up winning the battle. If I can't make the
linker do it, I'll do it myself!

This proved rather daunting. I started with Matt Pietrek's documentation
on the PE file format, and dug in all the source I could find on reading
.NET metadata. I finally settled down with the ECMA spec and coded up
some classes to read & modify metadata and tables.

In order to merge a .NET module, a CLR header has to be inserted into the
native DLL, and all the .NET metadata and IL as well. Then, since you're
not inserting it into the same relative places as they used to live, all
the RVA's have to be fixed up. This proved challenging.

Fixing up the metadata RVA's was easy. Having to write all the table
code to dig into the .NET metadata tables to fetch the MethodDef's and
FieldRVA's was a pain in the butt!

In the end, I got what I wanted. Source available here:
http://sqlite.phxsoftware.com/files/mergebin.zip

And that brings me to the reason for the post ...

The source isn't well documented yet -- I plan on writing up an article
or something for Codeguru in the near future. However, the code has some
caveats I'm hoping someone will know how to resolve ...

Caveats:
1. The native DLL must have an empty section in it large enough to hold
the .NET header, metadata and IL code. To create such a section, I used
this:
#pragma data_seg(\".clr\")
#pragma comment(linker, "/SECTION:.clr,ER")
char __ph[92316] = {0}; // 92316 is the number of bytes to reserve
#pragma data_seg()

2. The native DLL must have an exported function in it that calls
CorDllMain() in MSCOREE.DLL. The function name must have "CORDLLMAIN" in
it (search is case-insensitive) and must have the same calling
conventions as DllMain. The reason I require this is because I have to
remap the DLL EntryPoint to call this function, and set the entrypoint in
the .NET header to be the previous DLL entrypoint. The reason it must be
exported is because I don't know how to figure out the RVA of the
function without it being in the export table. Here's my stub:

extern BOOL WINAPI _CorDllMain(HINSTANCE, DWORD, LPVOID);
__declspec(dllexport) BOOL WINAPI _MyCorDllMain(HINSTANCE hInst,
DWORD dwReason,
LPVOID pvReserved)
{
return _CorDllMain(hInst, dwReason, pvReserved);
}

3. The code is largely untested -- I tested it for my own usage, but
your mileage may vary. Oh, if the .NET assembly is signed, you'll have
to re-sign it afterwards.

So that leads to the questions:

Is there a more graceful way to do this? I'd much rather expand the size
of the executable than have to hardcode a reserved space for the
assembly -- but at the same time I don't want to write out data into a
non-existent section of the file. Should I care? Will PEVerify
complain? It already complains about my non-standard stub.

Is there a way to set the entrypoint of the DLL directly to CorDllMain
rather than force you to write a stub that calls it?

Robert

 

[Robert Simpson]

For my purposes, no. You either make the developer install a native
(processor-specific) binary along with the wrapper, or you combine the two
libraries into a single file for each processor/platform. You still have to
include all the different processor builds in the cab file along with the
..NET wrapper anyway, so why NOT combine them?

There's no escaping the native DLL -- the wrapper has to have it. You also
need to avoid potential DLL hell involving the native DLL. If someone
overwrites the DLL with a newer version, there's no guarantee the old
wrapper will continue to function properly against it. The two are
inexorably tied.

It also simplifies maintenance and upgrades. One file means one less file
to patch, one less file that may be in use at the time, one less potential
problem. Also, since its a single file and self-contained, you know that
the wrapper is designed for and works with the exact build of the native
library it wraps.

On that note, maintenance is further simplified in that a change to one DLL
doesn't require full regression testing against every previous version of
the other DLL to ensure compatibility. By combining them I never have to
write patch notes that say "This updated wrapper only works on version X of
the native DLL"

Can you give me any compelling reason why installing two DLL's in your
application folder is better than one?

Robert

OK, so what is the advantage then? What are you winning in the end aside
from
being able to distribute one DLL and adding more complexity to the future
upgrades and maintainence.

--
Alex Yakhnin, .NET CF MVP
www.intelliprog.com | www.opennetcf.org

Really? I thought I was pretty clear.

I have a native DLL, and I have a .NET wrapper for it. I want to combine
them into a single DLL so that the DLL can be used both natively by
native
components, and the wrapper is used by managed components. In either
environment, the same DLL is used, and the wrapper P/Invokes into itself.

On the full framework, this is easy. You merely compile your .NET
wrapper
into a .netmodule and add a reference to the .netmodule as an input to
the
linker in the native DLL's project settings. You can't do this on the
Compact Framework though. The linker will throw all sorts of errors if
you
try.

Robert

Robert,

I am sorry, but I don't understand the purpose. What's the end goal
you're
trying to achieve?

--
Alex Yakhnin, .NET CF MVP
www.intelliprog.com | www.opennetcf.org

I've been searching high and low for the ability to do this on the
Compact Framework. In a nutshell, I have a 100% native DLL and I want
to
fold a 100% pure .NET assembly into it (the wrapper for it). At first
I
thought this would be pretty simple, since it is so easy to do on the
full framework. Boy was I wrong!

Originally I tried the old link a netmodule thing (adding a netmodule
to
the linker's INPUT parameter), but the CF linker either blew up or
gave
me an error. Works great on the full framework.

I tried lots and lots of combinations, but nothing worked. I
hand-edited
netmodules, poked the linker in uncomfortable ways, etc. No joy.

Sheer determination ended up winning the battle. If I can't make the
linker do it, I'll do it myself!

This proved rather daunting. I started with Matt Pietrek's
documentation
on the PE file format, and dug in all the source I could find on
reading
.NET metadata. I finally settled down with the ECMA spec and coded up
some classes to read & modify metadata and tables.

In order to merge a .NET module, a CLR header has to be inserted into
the
native DLL, and all the .NET metadata and IL as well. Then, since
you're
not inserting it into the same relative places as they used to live,
all
the RVA's have to be fixed up. This proved challenging.

Fixing up the metadata RVA's was easy. Having to write all the table
code to dig into the .NET metadata tables to fetch the MethodDef's and
FieldRVA's was a pain in the butt!

In the end, I got what I wanted. Source available here:
http://sqlite.phxsoftware.com/files/mergebin.zip

And that brings me to the reason for the post ...

The source isn't well documented yet -- I plan on writing up an
article
or something for Codeguru in the near future. However, the code has
some
caveats I'm hoping someone will know how to resolve ...

Caveats:
1. The native DLL must have an empty section in it large enough to
hold
the .NET header, metadata and IL code. To create such a section, I
used
this:
#pragma data_seg(\".clr\")
#pragma comment(linker, "/SECTION:.clr,ER")
char __ph[92316] = {0}; // 92316 is the number of bytes to reserve
#pragma data_seg()

2. The native DLL must have an exported function in it that calls
CorDllMain() in MSCOREE.DLL. The function name must have "CORDLLMAIN"
in
it (search is case-insensitive) and must have the same calling
conventions as DllMain. The reason I require this is because I have
to
remap the DLL EntryPoint to call this function, and set the entrypoint
in
the .NET header to be the previous DLL entrypoint. The reason it must
be
exported is because I don't know how to figure out the RVA of the
function without it being in the export table. Here's my stub:

extern BOOL WINAPI _CorDllMain(HINSTANCE, DWORD, LPVOID);
__declspec(dllexport) BOOL WINAPI _MyCorDllMain(HINSTANCE hInst,
DWORD dwReason,
LPVOID pvReserved)
{
return _CorDllMain(hInst, dwReason, pvReserved);
}

3. The code is largely untested -- I tested it for my own usage, but
your mileage may vary. Oh, if the .NET assembly is signed, you'll
have
to re-sign it afterwards.

So that leads to the questions:

Is there a more graceful way to do this? I'd much rather expand the
size
of the executable than have to hardcode a reserved space for the
assembly -- but at the same time I don't want to write out data into
a
non-existent section of the file. Should I care? Will PEVerify
complain? It already complains about my non-standard stub.

Is there a way to set the entrypoint of the DLL directly to CorDllMain
rather than force you to write a stub that calls it?

Robert