The implementation is the unit that needs to be tested. That is of course what you are instantiating and what contains the program/business logic.
If you had a critical interface and you wanted to make sure every implementation adhered to it properly, then you may write a test suite that focuses on the interface and requires an instance be passed in (agnostic of any implementation type).
Yes, it would probably be easier to use Mockito for PrintStream, it may not always be possible to avoid using a mock object like you did in this specific example.