Mixing a native DLL with a managed DLL on CF - solved

I've been searching high and low for the ability to do this on the Compact
Framework. In a nutshell, I have a 100% native DLL and I want to fold a
100% pure .NET assembly into it (the wrapper for it). At first I thought
this would be pretty simple, since it is so easy to do on the full
framework. Boy was I wrong!

Originally I tried the old link a netmodule thing (adding a netmodule to the
linker's INPUT parameter), but the CF linker either blew up or gave me an
error. Works great on the full framework.

I tried lots and lots of combinations, but nothing worked. I hand-edited
netmodules, poked the linker in uncomfortable ways, etc. No joy.

Sheer determination ended up winning the battle. If I can't make the linker
do it, I'll do it myself!

This proved rather daunting. I started with Matt Pietrek's documentation on
the PE file format, and dug in all the source I could find on reading .NET
metadata. I finally settled down with the ECMA spec and coded up some
classes to read & modify metadata and tables.

In order to merge a .NET module, a CLR header has to be inserted into the
native DLL, and all the .NET metadata and IL as well. Then, since you're
not inserting it into the same relative places as they used to live, all the
RVA's have to be fixed up. This proved challenging.

Fixing up the metadata RVA's was easy. Having to write all the table code
to dig into the .NET metadata tables to fetch the MethodDef's and FieldRVA's
was a pain in the butt!

In the end, I got what I wanted. Source available here:
http://sqlite.phxsoftware.com/files/mergebin.zip

And that brings me to the reason for the post ...

The source isn't well documented yet -- I plan on writing up an article or
something for Codeguru in the near future. However, the code has some
caveats I'm hoping someone will know how to resolve ...

Caveats:
1. The native DLL must have an empty section in it large enough to hold the
..NET header, metadata and IL code. To create such a section, I used this:
#pragma data_seg(\".clr\")
#pragma comment(linker, "/SECTION:.clr,ER")
char __ph[92316] = {0}; // 92316 is the number of bytes to reserve
#pragma data_seg()

2. The native DLL must have an exported function in it that calls
CorDllMain() in MSCOREE.DLL. The function name must have "CORDLLMAIN" in it
(search is case-insensitive) and must have the same calling conventions as
DllMain. The reason I require this is because I have to remap the DLL
EntryPoint to call this function, and set the entrypoint in the .NET header
to be the previous DLL entrypoint. The reason it must be exported is
because I don't know how to figure out the RVA of the function without it
being in the export table. Here's my stub:

extern BOOL WINAPI _CorDllMain(HINSTANCE, DWORD, LPVOID);
__declspec(dllexport) BOOL WINAPI _MyCorDllMain(HINSTANCE hInst,
DWORD dwReason,
LPVOID pvReserved)
{
return _CorDllMain(hInst, dwReason, pvReserved);
}

3. The code is largely untested -- I tested it for my own usage, but your
mileage may vary. Oh, if the .NET assembly is signed, you'll have to
re-sign it afterwards.

So that leads to the questions:

Is there a more graceful way to do this? I'd much rather expand the size of
the executable than have to hardcode a reserved space for the assembly --
but at the same time I don't want to write out data into a non-existent
section of the file. Should I care? Will PEVerify complain? It already
complains about my non-standard stub.

Is there a way to set the entrypoint of the DLL directly to CorDllMain
rather than force you to write a stub that calls it?

Robert

Robert,

I am sorry, but I don't understand the purpose. What's the end goal you're
trying to achieve?

--
Alex Yakhnin, .NET CF MVP
www.intelliprog.com | www.opennetcf.org

"Robert Simpson" <rmsimpson@noemail.noemail> wrote in message
news:uYxSTP7GGHA.648@TK2MSFTNGP14.phx.gbl...
> I've been searching high and low for the ability to do this on the Compact
> Framework. In a nutshell, I have a 100% native DLL and I want to fold a
> 100% pure .NET assembly into it (the wrapper for it). At first I thought
> this would be pretty simple, since it is so easy to do on the full
> framework. Boy was I wrong!
>
> Originally I tried the old link a netmodule thing (adding a netmodule to
> the linker's INPUT parameter), but the CF linker either blew up or gave me
> an error. Works great on the full framework.
>
> I tried lots and lots of combinations, but nothing worked. I hand-edited
> netmodules, poked the linker in uncomfortable ways, etc. No joy.
>
> Sheer determination ended up winning the battle. If I can't make the
> linker do it, I'll do it myself!
>
> This proved rather daunting. I started with Matt Pietrek's documentation
> on the PE file format, and dug in all the source I could find on reading
> .NET metadata. I finally settled down with the ECMA spec and coded up
> some classes to read & modify metadata and tables.
>
> In order to merge a .NET module, a CLR header has to be inserted into the
> native DLL, and all the .NET metadata and IL as well. Then, since you're
> not inserting it into the same relative places as they used to live, all
> the RVA's have to be fixed up. This proved challenging.
>
> Fixing up the metadata RVA's was easy. Having to write all the table code
> to dig into the .NET metadata tables to fetch the MethodDef's and
> FieldRVA's was a pain in the butt!
>
> In the end, I got what I wanted. Source available here:
> http://sqlite.phxsoftware.com/files/mergebin.zip
>
> And that brings me to the reason for the post ...
>
> The source isn't well documented yet -- I plan on writing up an article or
> something for Codeguru in the near future. However, the code has some
> caveats I'm hoping someone will know how to resolve ...
>
> Caveats:
> 1. The native DLL must have an empty section in it large enough to hold
> the .NET header, metadata and IL code. To create such a section, I used
> this:
> #pragma data_seg(\".clr\")
> #pragma comment(linker, "/SECTION:.clr,ER")
> char __ph[92316] = {0}; // 92316 is the number of bytes to reserve
> #pragma data_seg()
>
> 2. The native DLL must have an exported function in it that calls
> CorDllMain() in MSCOREE.DLL. The function name must have "CORDLLMAIN" in
> it (search is case-insensitive) and must have the same calling conventions
> as DllMain. The reason I require this is because I have to remap the DLL
> EntryPoint to call this function, and set the entrypoint in the .NET
> header to be the previous DLL entrypoint. The reason it must be exported
> is because I don't know how to figure out the RVA of the function without
> it being in the export table. Here's my stub:
>
> extern BOOL WINAPI _CorDllMain(HINSTANCE, DWORD, LPVOID);
> __declspec(dllexport) BOOL WINAPI _MyCorDllMain(HINSTANCE hInst,
> DWORD dwReason,
> LPVOID pvReserved)
> {
> return _CorDllMain(hInst, dwReason, pvReserved);
> }
>
> 3. The code is largely untested -- I tested it for my own usage, but your
> mileage may vary. Oh, if the .NET assembly is signed, you'll have to
> re-sign it afterwards.
>
> So that leads to the questions:
>
> Is there a more graceful way to do this? I'd much rather expand the size
> of the executable than have to hardcode a reserved space for the
> assembly -- but at the same time I don't want to write out data into a
> non-existent section of the file. Should I care? Will PEVerify complain?
> It already complains about my non-standard stub.
>
> Is there a way to set the entrypoint of the DLL directly to CorDllMain
> rather than force you to write a stub that calls it?
>
> Robert
>
>

Really? I thought I was pretty clear.

I have a native DLL, and I have a .NET wrapper for it. I want to combine
them into a single DLL so that the DLL can be used both natively by native
components, and the wrapper is used by managed components. In either
environment, the same DLL is used, and the wrapper P/Invokes into itself.

On the full framework, this is easy. You merely compile your .NET wrapper
into a .netmodule and add a reference to the .netmodule as an input to the
linker in the native DLL's project settings. You can't do this on the
Compact Framework though. The linker will throw all sorts of errors if you
try.

Robert

"Alex Yakhnin [MVP]" <a.yakhnin@online.att.net> wrote in message
news:uBsUvH%23GGHA.2300@TK2MSFTNGP15.phx.gbl...
> Robert,
>
> I am sorry, but I don't understand the purpose. What's the end goal you're
> trying to achieve?
>
> --
> Alex Yakhnin, .NET CF MVP
> www.intelliprog.com | www.opennetcf.org
>
> "Robert Simpson" <rmsimpson@noemail.noemail> wrote in message
> news:uYxSTP7GGHA.648@TK2MSFTNGP14.phx.gbl...
>> I've been searching high and low for the ability to do this on the
>> Compact Framework. In a nutshell, I have a 100% native DLL and I want to
>> fold a 100% pure .NET assembly into it (the wrapper for it). At first I
>> thought this would be pretty simple, since it is so easy to do on the
>> full framework. Boy was I wrong!
>>
>> Originally I tried the old link a netmodule thing (adding a netmodule to
>> the linker's INPUT parameter), but the CF linker either blew up or gave
>> me an error. Works great on the full framework.
>>
>> I tried lots and lots of combinations, but nothing worked. I hand-edited
>> netmodules, poked the linker in uncomfortable ways, etc. No joy.
>>
>> Sheer determination ended up winning the battle. If I can't make the
>> linker do it, I'll do it myself!
>>
>> This proved rather daunting. I started with Matt Pietrek's documentation
>> on the PE file format, and dug in all the source I could find on reading
>> .NET metadata. I finally settled down with the ECMA spec and coded up
>> some classes to read & modify metadata and tables.
>>
>> In order to merge a .NET module, a CLR header has to be inserted into the
>> native DLL, and all the .NET metadata and IL as well. Then, since you're
>> not inserting it into the same relative places as they used to live, all
>> the RVA's have to be fixed up. This proved challenging.
>>
>> Fixing up the metadata RVA's was easy. Having to write all the table
>> code to dig into the .NET metadata tables to fetch the MethodDef's and
>> FieldRVA's was a pain in the butt!
>>
>> In the end, I got what I wanted. Source available here:
>> http://sqlite.phxsoftware.com/files/mergebin.zip
>>
>> And that brings me to the reason for the post ...
>>
>> The source isn't well documented yet -- I plan on writing up an article
>> or something for Codeguru in the near future. However, the code has some
>> caveats I'm hoping someone will know how to resolve ...
>>
>> Caveats:
>> 1. The native DLL must have an empty section in it large enough to hold
>> the .NET header, metadata and IL code. To create such a section, I used
>> this:
>> #pragma data_seg(\".clr\")
>> #pragma comment(linker, "/SECTION:.clr,ER")
>> char __ph[92316] = {0}; // 92316 is the number of bytes to reserve
>> #pragma data_seg()
>>
>> 2. The native DLL must have an exported function in it that calls
>> CorDllMain() in MSCOREE.DLL. The function name must have "CORDLLMAIN" in
>> it (search is case-insensitive) and must have the same calling
>> conventions as DllMain. The reason I require this is because I have to
>> remap the DLL EntryPoint to call this function, and set the entrypoint in
>> the .NET header to be the previous DLL entrypoint. The reason it must be
>> exported is because I don't know how to figure out the RVA of the
>> function without it being in the export table. Here's my stub:
>>
>> extern BOOL WINAPI _CorDllMain(HINSTANCE, DWORD, LPVOID);
>> __declspec(dllexport) BOOL WINAPI _MyCorDllMain(HINSTANCE hInst,
>> DWORD dwReason,
>> LPVOID pvReserved)
>> {
>> return _CorDllMain(hInst, dwReason, pvReserved);
>> }
>>
>> 3. The code is largely untested -- I tested it for my own usage, but
>> your mileage may vary. Oh, if the .NET assembly is signed, you'll have
>> to re-sign it afterwards.
>>
>> So that leads to the questions:
>>
>> Is there a more graceful way to do this? I'd much rather expand the size
>> of the executable than have to hardcode a reserved space for the
>> assembly -- but at the same time I don't want to write out data into a
>> non-existent section of the file. Should I care? Will PEVerify
>> complain? It already complains about my non-standard stub.
>>
>> Is there a way to set the entrypoint of the DLL directly to CorDllMain
>> rather than force you to write a stub that calls it?
>>
>> Robert
>>
>>
>
>

OK, so what is the advantage then? What are you winning in the end aside from
being able to distribute one DLL and adding more complexity to the future
upgrades and maintainence.

--
Alex Yakhnin, .NET CF MVP
www.intelliprog.com | www.opennetcf.org


"Robert Simpson" wrote:

> Really? I thought I was pretty clear.
>
> I have a native DLL, and I have a .NET wrapper for it. I want to combine
> them into a single DLL so that the DLL can be used both natively by native
> components, and the wrapper is used by managed components. In either
> environment, the same DLL is used, and the wrapper P/Invokes into itself.
>
> On the full framework, this is easy. You merely compile your .NET wrapper
> into a .netmodule and add a reference to the .netmodule as an input to the
> linker in the native DLL's project settings. You can't do this on the
> Compact Framework though. The linker will throw all sorts of errors if you
> try.
>
> Robert
>
> "Alex Yakhnin [MVP]" <a.yakhnin@online.att.net> wrote in message
> news:uBsUvH%23GGHA.2300@TK2MSFTNGP15.phx.gbl...
> > Robert,
> >
> > I am sorry, but I don't understand the purpose. What's the end goal you're
> > trying to achieve?
> >
> > --
> > Alex Yakhnin, .NET CF MVP
> > www.intelliprog.com | www.opennetcf.org
> >
> > "Robert Simpson" <rmsimpson@noemail.noemail> wrote in message
> > news:uYxSTP7GGHA.648@TK2MSFTNGP14.phx.gbl...
> >> I've been searching high and low for the ability to do this on the
> >> Compact Framework. In a nutshell, I have a 100% native DLL and I want to
> >> fold a 100% pure .NET assembly into it (the wrapper for it). At first I
> >> thought this would be pretty simple, since it is so easy to do on the
> >> full framework. Boy was I wrong!
> >>
> >> Originally I tried the old link a netmodule thing (adding a netmodule to
> >> the linker's INPUT parameter), but the CF linker either blew up or gave
> >> me an error. Works great on the full framework.
> >>
> >> I tried lots and lots of combinations, but nothing worked. I hand-edited
> >> netmodules, poked the linker in uncomfortable ways, etc. No joy.
> >>
> >> Sheer determination ended up winning the battle. If I can't make the
> >> linker do it, I'll do it myself!
> >>
> >> This proved rather daunting. I started with Matt Pietrek's documentation
> >> on the PE file format, and dug in all the source I could find on reading
> >> .NET metadata. I finally settled down with the ECMA spec and coded up
> >> some classes to read & modify metadata and tables.
> >>
> >> In order to merge a .NET module, a CLR header has to be inserted into the
> >> native DLL, and all the .NET metadata and IL as well. Then, since you're
> >> not inserting it into the same relative places as they used to live, all
> >> the RVA's have to be fixed up. This proved challenging.
> >>
> >> Fixing up the metadata RVA's was easy. Having to write all the table
> >> code to dig into the .NET metadata tables to fetch the MethodDef's and
> >> FieldRVA's was a pain in the butt!
> >>
> >> In the end, I got what I wanted. Source available here:
> >> http://sqlite.phxsoftware.com/files/mergebin.zip
> >>
> >> And that brings me to the reason for the post ...
> >>
> >> The source isn't well documented yet -- I plan on writing up an article
> >> or something for Codeguru in the near future. However, the code has some
> >> caveats I'm hoping someone will know how to resolve ...
> >>
> >> Caveats:
> >> 1. The native DLL must have an empty section in it large enough to hold
> >> the .NET header, metadata and IL code. To create such a section, I used
> >> this:
> >> #pragma data_seg(\".clr\")
> >> #pragma comment(linker, "/SECTION:.clr,ER")
> >> char __ph[92316] = {0}; // 92316 is the number of bytes to reserve
> >> #pragma data_seg()
> >>
> >> 2. The native DLL must have an exported function in it that calls
> >> CorDllMain() in MSCOREE.DLL. The function name must have "CORDLLMAIN" in
> >> it (search is case-insensitive) and must have the same calling
> >> conventions as DllMain. The reason I require this is because I have to
> >> remap the DLL EntryPoint to call this function, and set the entrypoint in
> >> the .NET header to be the previous DLL entrypoint. The reason it must be
> >> exported is because I don't know how to figure out the RVA of the
> >> function without it being in the export table. Here's my stub:
> >>
> >> extern BOOL WINAPI _CorDllMain(HINSTANCE, DWORD, LPVOID);
> >> __declspec(dllexport) BOOL WINAPI _MyCorDllMain(HINSTANCE hInst,
> >> DWORD dwReason,
> >> LPVOID pvReserved)
> >> {
> >> return _CorDllMain(hInst, dwReason, pvReserved);
> >> }
> >>
> >> 3. The code is largely untested -- I tested it for my own usage, but
> >> your mileage may vary. Oh, if the .NET assembly is signed, you'll have
> >> to re-sign it afterwards.
> >>
> >> So that leads to the questions:
> >>
> >> Is there a more graceful way to do this? I'd much rather expand the size
> >> of the executable than have to hardcode a reserved space for the
> >> assembly -- but at the same time I don't want to write out data into a
> >> non-existent section of the file. Should I care? Will PEVerify
> >> complain? It already complains about my non-standard stub.
> >>
> >> Is there a way to set the entrypoint of the DLL directly to CorDllMain
> >> rather than force you to write a stub that calls it?
> >>
> >> Robert
> >>
> >>
> >
> >
>
>
>

For my purposes, no. You either make the developer install a native
(processor-specific) binary along with the wrapper, or you combine the two
libraries into a single file for each processor/platform. You still have to
include all the different processor builds in the cab file along with the
..NET wrapper anyway, so why NOT combine them?

There's no escaping the native DLL -- the wrapper has to have it. You also
need to avoid potential DLL hell involving the native DLL. If someone
overwrites the DLL with a newer version, there's no guarantee the old
wrapper will continue to function properly against it. The two are
inexorably tied.

It also simplifies maintenance and upgrades. One file means one less file
to patch, one less file that may be in use at the time, one less potential
problem. Also, since its a single file and self-contained, you know that
the wrapper is designed for and works with the exact build of the native
library it wraps.

On that note, maintenance is further simplified in that a change to one DLL
doesn't require full regression testing against every previous version of
the other DLL to ensure compatibility. By combining them I never have to
write patch notes that say "This updated wrapper only works on version X of
the native DLL"

Can you give me any compelling reason why installing two DLL's in your
application folder is better than one?

Robert


"Alex Yakhnin [MVP]" <a.yakhnin@online.att.net> wrote in message
news:1E7249A0-A6F7-4177-80D5-F6D535C3ADE1@microsoft.com...
> OK, so what is the advantage then? What are you winning in the end aside
> from
> being able to distribute one DLL and adding more complexity to the future
> upgrades and maintainence.
>
> --
> Alex Yakhnin, .NET CF MVP
> www.intelliprog.com | www.opennetcf.org
>
>
> "Robert Simpson" wrote:
>
>> Really? I thought I was pretty clear.
>>
>> I have a native DLL, and I have a .NET wrapper for it. I want to combine
>> them into a single DLL so that the DLL can be used both natively by
>> native
>> components, and the wrapper is used by managed components. In either
>> environment, the same DLL is used, and the wrapper P/Invokes into itself.
>>
>> On the full framework, this is easy. You merely compile your .NET
>> wrapper
>> into a .netmodule and add a reference to the .netmodule as an input to
>> the
>> linker in the native DLL's project settings. You can't do this on the
>> Compact Framework though. The linker will throw all sorts of errors if
>> you
>> try.
>>
>> Robert
>>
>> "Alex Yakhnin [MVP]" <a.yakhnin@online.att.net> wrote in message
>> news:uBsUvH%23GGHA.2300@TK2MSFTNGP15.phx.gbl...
>> > Robert,
>> >
>> > I am sorry, but I don't understand the purpose. What's the end goal
>> > you're
>> > trying to achieve?
>> >
>> > --
>> > Alex Yakhnin, .NET CF MVP
>> > www.intelliprog.com | www.opennetcf.org
>> >
>> > "Robert Simpson" <rmsimpson@noemail.noemail> wrote in message
>> > news:uYxSTP7GGHA.648@TK2MSFTNGP14.phx.gbl...
>> >> I've been searching high and low for the ability to do this on the
>> >> Compact Framework. In a nutshell, I have a 100% native DLL and I want
>> >> to
>> >> fold a 100% pure .NET assembly into it (the wrapper for it). At first
>> >> I
>> >> thought this would be pretty simple, since it is so easy to do on the
>> >> full framework. Boy was I wrong!
>> >>
>> >> Originally I tried the old link a netmodule thing (adding a netmodule
>> >> to
>> >> the linker's INPUT parameter), but the CF linker either blew up or
>> >> gave
>> >> me an error. Works great on the full framework.
>> >>
>> >> I tried lots and lots of combinations, but nothing worked. I
>> >> hand-edited
>> >> netmodules, poked the linker in uncomfortable ways, etc. No joy.
>> >>
>> >> Sheer determination ended up winning the battle. If I can't make the
>> >> linker do it, I'll do it myself!
>> >>
>> >> This proved rather daunting. I started with Matt Pietrek's
>> >> documentation
>> >> on the PE file format, and dug in all the source I could find on
>> >> reading
>> >> .NET metadata. I finally settled down with the ECMA spec and coded up
>> >> some classes to read & modify metadata and tables.
>> >>
>> >> In order to merge a .NET module, a CLR header has to be inserted into
>> >> the
>> >> native DLL, and all the .NET metadata and IL as well. Then, since
>> >> you're
>> >> not inserting it into the same relative places as they used to live,
>> >> all
>> >> the RVA's have to be fixed up. This proved challenging.
>> >>
>> >> Fixing up the metadata RVA's was easy. Having to write all the table
>> >> code to dig into the .NET metadata tables to fetch the MethodDef's and
>> >> FieldRVA's was a pain in the butt!
>> >>
>> >> In the end, I got what I wanted. Source available here:
>> >> http://sqlite.phxsoftware.com/files/mergebin.zip
>> >>
>> >> And that brings me to the reason for the post ...
>> >>
>> >> The source isn't well documented yet -- I plan on writing up an
>> >> article
>> >> or something for Codeguru in the near future. However, the code has
>> >> some
>> >> caveats I'm hoping someone will know how to resolve ...
>> >>
>> >> Caveats:
>> >> 1. The native DLL must have an empty section in it large enough to
>> >> hold
>> >> the .NET header, metadata and IL code. To create such a section, I
>> >> used
>> >> this:
>> >> #pragma data_seg(\".clr\")
>> >> #pragma comment(linker, "/SECTION:.clr,ER")
>> >> char __ph[92316] = {0}; // 92316 is the number of bytes to reserve
>> >> #pragma data_seg()
>> >>
>> >> 2. The native DLL must have an exported function in it that calls
>> >> CorDllMain() in MSCOREE.DLL. The function name must have "CORDLLMAIN"
>> >> in
>> >> it (search is case-insensitive) and must have the same calling
>> >> conventions as DllMain. The reason I require this is because I have
>> >> to
>> >> remap the DLL EntryPoint to call this function, and set the entrypoint
>> >> in
>> >> the .NET header to be the previous DLL entrypoint. The reason it must
>> >> be
>> >> exported is because I don't know how to figure out the RVA of the
>> >> function without it being in the export table. Here's my stub:
>> >>
>> >> extern BOOL WINAPI _CorDllMain(HINSTANCE, DWORD, LPVOID);
>> >> __declspec(dllexport) BOOL WINAPI _MyCorDllMain(HINSTANCE hInst,
>> >> DWORD dwReason,
>> >> LPVOID pvReserved)
>> >> {
>> >> return _CorDllMain(hInst, dwReason, pvReserved);
>> >> }
>> >>
>> >> 3. The code is largely untested -- I tested it for my own usage, but
>> >> your mileage may vary. Oh, if the .NET assembly is signed, you'll
>> >> have
>> >> to re-sign it afterwards.
>> >>
>> >> So that leads to the questions:
>> >>
>> >> Is there a more graceful way to do this? I'd much rather expand the
>> >> size
>> >> of the executable than have to hardcode a reserved space for the
>> >> assembly -- but at the same time I don't want to write out data into
>> >> a
>> >> non-existent section of the file. Should I care? Will PEVerify
>> >> complain? It already complains about my non-standard stub.
>> >>
>> >> Is there a way to set the entrypoint of the DLL directly to CorDllMain
>> >> rather than force you to write a stub that calls it?
>> >>
>> >> Robert
>> >>
>> >>
>> >
>> >
>>
>>
>>