There is a way to do this, but you will have to hand edit your project files. The project files can have a Condition attribute applied to them in many of the elements, including the one for references.
You can add these to your references to specify when the reference should be used:
<Reference Include="Product, Version=1.0.0.0" Condition="'$(Configuration)'=='V1'">
</Reference>
<Reference Include="Product, Version=2.0.0.0" Condition="'$(Configuration)'=='V2'">
</Reference>
<Reference Include="Product, Version=3.0.0.0" Condition="'$(Configuration)'=='V3'">
</Reference>
You then define several build configurations (V1, V2, V3) and each reference will be included only in the relevant chosen build configuration.
Combine this with conditional compilation symbols and #if statements in your code and you should be able to do what you want.
A thing to be careful of if you do this is that it is easy to have Visual Studio remove the conditional attributes from the project file.