.config file if app directory moves?

Hi all...

I have a VB.NET project that compiles an .exe (for example test.exe)
Visual Studio locates this .exe in a specific directory. If I create
in this same directory a config file test.exe.config, and run test.exe
here, everything works fine and the settings in test.exe.config are
picked up just fine by test.exe

However, if I take test.exe and test.exe.config and move them to a
completely different path, then when I run test.exe it does *not* pick
up the settings in test.exe.config, even though they continue to be
colocated in the same directory.

What is going on here? how can I get my app to successfuly use a
..config file even if I move the .exe (and the config file) to a new
path?

Thanks in advance...