I don’t see anything wrong with using ConditionalWeakTable. If you need ephemerons, you pretty much have no other choice.
I don’t think future .NET versions will be a problem – even if only compilers would use this class, Microsoft still couldn’t change it without breaking compatibility with existing binaries.
As for overhead – there certainly will be overhead compared to a normal Dictionary. Having many DependentHandles probably will be expensive similarly to how many WeakReferences are more expensive than normal references (the GC has to do additional work to scan them to see if they need to be nulled out). But that’s not a problem unless you have lots (several million) of entries.