So Visual Studio uses a framework called Visual Studio Test Platform to load test adapters,
https://github.com/Microsoft/vstest
To discover or execute test cases, VSTest would call the test adapters based on your project configuration. (That’s why NUnit/xUnit/MSTest all ask you to install a test adapter NuGet package to your unit testing projects). So MSTest.TestAdapter exists for that purposes.
MSTest.TestFramework itself implements the testing frameworks and its contracts. So you need to add a NuGet reference of it in order to write unit test cases and have them compiled. Only compiled projects along with the test adapter can then be consumed by Visual Studio.
So the final answer to your question would be “you usually need both”.
The other answer from @Nkosi of course can be right if you don’t ever plan to use Visual Studio. MSTest has its own command line runner, which can run your unit test project without the test adapter.