I think you nailed the two compelling reasons 🙂
The MemoryCache has an eviction strategy, so that it can throw out entries that are no longer needed or for that you do not have enough memory anymore.
A Dictionary will not “lose contents”.
Update: MemoryCache is thread-safe and has methods such as AddOrGetExisting. With a Dictionary, you’d have to synchronize access yourself (or use ConcurrentDictionary).