You don’t indicate what the error was, but this should work:
unitOfWork.Setup(m => m.PhysicalTests).Returns((IRepository<PhysicalTest>)null);
I suspect you tried to call it with Returns(null), which causes the compiler to complain since Returns is overloaded and it doesn’t know which method should be called. Casting to a specific type removes the ambiguity.