Build-dependent references

I have a C# DLL that references a mixed-mode C++ DLL of mine. The mixed-mode
DLL compiles either to a debug version or a release version. How can I
specify in my C# project that I want the reference to be to the debug version
of the C++ DLL for the debug build of the C# project, and likewise for
release? I have tried all the avenues I could think of. Surely there must be
a way? Right now I have to go and change the references every time I want to
switch between debug/release testing.

Thanks,
Ben