In case the exception is very specific to Foo class, I don’t mind keeping it as a public Nested class. And whenever it seems like a time to extract that out, simply extract that out.
In general practice, I have never seen any nested class defined for Exception, though. I also don’t know if one exist in Java API.