C++ exe and library search paths

P

paul-shed

Hello,

Using C++ under GNU Linux I can create a library or exectuable that contains
a relative search path (rpath) to search for other shared librarires (dlls).
The loader then uses this relative path before it searches via other means
(PATH env variable etc). This is great as I don't have to gum up my
application directory with other dlls or plugins, I can push them down into a
separate directories.

Now is there any way I can do this in VS2008 using assemblies/manifest
files? I want to specify relative paths that the loader should search for
dependent dlls.

At the moment I have to add these paths to the PATHs variable and we all
know how much of a pain that is to maintain.

Regards,

paul-shed
 
J

Jialiang Ge [MSFT]

Good morning paul-shed. Welcome to the VC newsgroup! My name is Jialiang Ge
[MSFT]. It's my pleasure to work with you on this issue.

The question is about how to specify the relative paths that the loader
should search for the dependent DLLs in Windows. I'd like to first share
several solutions that quickly emerge in my mind when I see the question. I
will research for better ones (for example, some easier settings in a
manifest file) and report back as soon as possible.

=================================================
Solution 1. Using the codebase or probe element in the configuration file

This solution only works for .NET assemblies. Because your question is
posted in dotnet.language.vc queue, I assume that your are working with
C++/CLI dll and exe. If I'm wrong, please skip this solution.

The document of the codebase element can be found at
http://msdn.microsoft.com/en-us/library/efs781xb.aspx
The codeBase element (<codeBase>) is an element used to configure an
assembly DLL that allows to download the DLL on demand from cross the
network the first time it's used by hosting application. We can set the
href value to be a path relative to the application's directory, and the
.NET runtime will search the path for the target DLL.

The document of the probe element can be found at
http://msdn.microsoft.com/en-us/library/823z9h8w.aspx
The probing element(<probing>) specifies application base subdirectories
for CLR to search when loading assemblies. This element has an attribute
called "privatePath" which specifies subdirectories of the application's
base directory that might contain assemblies. Delimit each subdirectory
with a semicolon.


=================================================
Solution 2. Delay-loading the Dlls and use the API SetDllDirectory

Generally speaking, there are three ways to load a DLL into a process.

1. Implicitly Loading (and linking) the required DLL. In this way, the
application's source code simply references symbols contained in the DLL,
and the DLL is loaded when the application is invoked.
http://msdn.microsoft.com/en-us/library/ms684184(VS.85).aspx

2. Explicitly loading the Dll with the API LoadLibrary(Ex). We can specify
the relative path to the target DLL in the lpFileName parameter, so that
the process will search the path we wanted.
http://msdn.microsoft.com/en-us/library/ms685090(VS.85).aspx

3. Delay loading the DLL.
A delay-load DLL is a DLL that is implicitly linked but not actually loaded
until your code attempts to reference a symbol contained within the DLL.

According to your post description, what you currently have is the approach
1 (Implicitly loading the Dll). Its DLL search order is
http://msdn.microsoft.com/en-us/library/ms682586(VS.85).aspx. Without a
manifest file or DLL redirection, the loader searches for the
user-specified location only when we call SetDllDirectory. SetDllDirectory
cannot be used in the approach 1 because the load of the DLLs happens
before the execution of our own codes. The solution is delay-loading the
DLLs.

Step1. In the project property page of the client app, switch to
Linker->Input0>Delay Loaded DLLs, and input the DLLs that are not in the
app folder into the textbox. These DLLs won't be loaded in the load time.
The DLL is loaded when its exported element is consumed by the client app.

Step2. If you want to allow explicit unloading of the delayed load DLLs,
please turn to the page Linker->Advanced->Delay Loaded DLL, and change the
value to Support Unload (/DELAY:UNLOAD)

Step3. In the main entry of the client app, add this code at the beginning:
SetDllDirectory(L"[relative path]");
In this way, when the DLLs in the relative path are consumed, the loader
will search for the DLL in the relative path specified in SetDllDirectory.

Please let me know whether you like this idea or not. Delay loading a DLL
has many other benefits. For example, if the application uses a lot of
DLLs, the initialization time will be slow because the loader maps all the
required DLLs into the process' address space. Delay-load DLLs spreads out
the loading of the DLLs as the process executes, thus, the initialization
will have better performance. In addition, delay-loading DLLs gives us a
chance to decide whether or not we really should/need to load a certain DLL
when the DLL is not compatible with the current environment.


You may want to try the above two solutions first. I'm looking for some
settings in the app's manifest file that can accomplish the task. I see an
element <file> that may be useful. I'm studying the use of it.

Best Regards,
Jialiang Ge ([email protected], remove 'online.')
Microsoft Online Community Support

Delighting our customers is our #1 priority. We welcome your comments and
suggestions about how we can improve the support we provide to you. Please
feel free to let my manager know what you think of the level of service
provided. You can send feedback directly to my manager at:
(e-mail address removed).

==================================================
Get notification to my posts through email? Please refer to
http://msdn.microsoft.com/en-us/subscriptions/aa948868.aspx#notifications.

MSDN Managed Newsgroup support offering is for non-urgent issues where an
initial response from the community or a Microsoft Support Engineer within
2 business day is acceptable. Please note that each follow up response may
take approximately 2 business days as the support professional working with
you may need further investigation to reach the most efficient resolution.
The offering is not appropriate for situations that require urgent,
real-time or phone-based interactions. Issues of this nature are best
handled working with a dedicated Microsoft Support Engineer by contacting
Microsoft Customer Support Services (CSS) at
http://msdn.microsoft.com/en-us/subscriptions/aa948874.aspx
==================================================
This posting is provided "AS IS" with no warranties, and confers no rights.
 
J

Jialiang Ge [MSFT]

Hello paul-shed

I'd like to add two more solutions for your references. They are the
evolvement of 'Solution 2' in my first reply.

====================================
Solution 3.
After we configured the delay-load of the DLL, we can call LoadLibrary(
fullPathToDll ) to pro-actively load the module, instead of
SetDllDirectory, before calling anything in that DLL.

====================================
Solution 4.
If the DLL can be delay-loaded, then another option is to set a
notification hook. http://msdn.microsoft.com/en-us/library/z9h1h6ty.aspx.
In the dliNotePreLoadLibrary notification implementation, when the szDll
member of the DelayLoadInfo structure is the name of the DLL you want, you
can LoadLibrary the DLL from wherever you want and return the HMODULE
(return 0 for all other notifications). We can construct the path to be
passed to LoadLibrary by using GetModuleFileName(NULL, ...), which will
return the full path to the running exe.


I have little progress in the direction of a manifest setting. I will
perform more researches to see whether or not the manifest file supports
specifying the search path of a DLL. (Note: I'm talking about native C++
here. If you are writing managed code, please refer to Solution 1 in my
first reply)

Regards,
Jialiang Ge
Microsoft Online Community Support

=================================================
Delighting our customers is our #1 priority. We welcome your comments and
suggestions about how we can improve the support we provide to you. Please
feel free to let my manager know what you think of the level of service
provided. You can send feedback directly to my manager at:
(e-mail address removed).

This posting is provided "AS IS" with no warranties, and confers no rights.
=================================================
 
P

paul-shed

Jialiang,

Thanks for these replies.

Yes I am using unmanaged code. I'm using a mixture of implicit loading and
explicit loading through loadlibrary. I'm using some Third Party dlls in my
application which simply can't be delay loaded because to get to main
(WinMain) my application relies on them being loaded.

So that's why I really want a solution that's similar to the GNU/Elf rpath
model. I would have thought that an assembly/manifest would be able to do
this.

Regards,

paul-shed
 
J

Jialiang Ge [MSFT]

Hello paul-shed

I understand your concerns. I discussed the manifest setting with the
Windows team, and got this information: Using an application manifest to
achieve this kind of behavior is a Windows 7 feature, where probing
privatePath support has been added for native assemblies as well. However,
in the current Windows systems, the feature is not available.

I will try my best to find some workarounds for you. It might be helpful if
you could provide more background info of:

<quote>
Some Third Party dlls in my application which simply can't be delay loaded
because to get to main (WinMain) my application relies on them being loaded.
</quote>

If it is not proper to tell the info in the community, please feel free to
email me: (e-mail address removed)

Regards,
Jialiang Ge ([email protected], remove 'online.')
Microsoft Online Community Support

=================================================
Delighting our customers is our #1 priority. We welcome your comments and
suggestions about how we can improve the support we provide to you. Please
feel free to let my manager know what you think of the level of service
provided. You can send feedback directly to my manager at:
(e-mail address removed).

This posting is provided "AS IS" with no warranties, and confers no rights.
=================================================
 
P

paul-shed

Jialiang

Thanks for this. I've given this some more thought this morning with my
application, now that I know it can't be done using assemblies/manifests.

I think I may be able to reorganize the loading to use delay loading of my
dependent dlls. I'll go and try that now.

Essentially you've answered my question wo se'll consider this closed.

Regards,

paul- shed
 
J

Jialiang Ge [MSFT]

Hello paul-shed

I'm going to continue researching the subject for one day. If I have any
new finding, I will update you.

Regards,
Jialiang Ge ([email protected], remove 'online.')
Microsoft Online Community Support

=================================================
Delighting our customers is our #1 priority. We welcome your comments and
suggestions about how we can improve the support we provide to you. Please
feel free to let my manager know what you think of the level of service
provided. You can send feedback directly to my manager at:
(e-mail address removed).

This posting is provided "AS IS" with no warranties, and confers no rights.
=================================================
 
J

Jialiang Ge [MSFT]

Hello paul- shed

How's your trial of reorganizing the loading to use delay-load of the
dependent DLLs? Are you getting along well with the solution? If you meet
with any difficulty, please feel free to tell me.

Below, I list several difficulties that you may possibly be faced with.

=======================================
1. You may have some statically-linked-DLLs, and we want to re-organize the
loading order so that the first DLL is loaded first.

Solution:
a. These "other" statically linked DLLs should also be delay-loaded so that
the EXE can load the first DLL first.
b. The notification hook can specify the location of the DLL.

=======================================
2. Maybe you have global C++ objects in the exe that need to be constructed
before CRT calls winmain. For example

int g_val = Add(1, 2); // Add is exported by a DLL
int _tmain(int argc, _TCHAR* argv[])
{
......
}

Delay-load should handle this fine, however, my solution 2 and 3 do not
help. We need to use the notification hook (the solution 4).

/delayload is compiler-generated and the callback is discovered by the
compiler because you implement it, not because you register it at runtime.
So even if a global C++ object is constructed before main/winmain, any
delayload functions it calls will go through the same delay-load
implementation. The only caveat is dealing with the order of initialization
of global variables across separate translation units. In the EXE's code,
you have to ensure that the global __pfnDliNotifyHook2 function pointer is
initialized before any exported function from the DLL is called as a result
of the dynamic initialization of some other global variable. The best way
to ensure that the delay load hook is initialized first would be to specify
#pragma init_seg(lib) in the TU that initializes __pfnDliNotifyHook2.

In case that you want an example of notification hook, here is one:
http://www.koders.com/cpp/fid6654B9013A5F136F216CB3D1652CEAC3F6BEBCA4.aspx.

=======================================
3. The DLL needs to do "something else" inside the DllMain that affects the
process. Maybe this is something unsupported like modifying the process or
API hooking.

This will be difficult (and unsupported). Fusion can't use relative paths
(before win7) and Windows doesn't have private DLL searching other than
fusion. You might need a "wrapper EXE" that modifies the PATH before
calling the real exe. This isn't a great solution but it would work.


I hope that the above info is useful to you. Again, if you meet with any
difficulty, please feel free to tell me. I'm always with you.

Regards,
Jialiang Ge ([email protected], remove 'online.')
Microsoft Online Community Support

=================================================
Delighting our customers is our #1 priority. We welcome your comments and
suggestions about how we can improve the support we provide to you. Please
feel free to let my manager know what you think of the level of service
provided. You can send feedback directly to my manager at:
(e-mail address removed).

This posting is provided "AS IS" with no warranties, and confers no rights.
=================================================
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Top