Like your own question suggests, the key is the domain, not JPA. JPA is just a framework which you can (and should) use in a way which best fits your problem. Choosing a suboptimal solution because of framework (or its limits) is usually a warning bell.
When I need a set and never care about order, I use a Set. When for some reason order is important (ordered list, ordering by date, etc.), then a List.
You seem to be well aware of the difference between Collection, Set, and List. The only reason to use one vs. the other depends only on your needs. You can use them to communicate to users of your API (or your future self) the properties of your collection (which may be subtle or implicit).
This is follows the exact same rules as using different collection types anywhere else throughout your code. You could use Object or Collections for all your references, yet in most cases you use more concrete types.
For example, when I see a List, I know it comes sorted in some way, and that duplicates are either acceptable or irrelevant for this case. When I see a Set, I usually expect it to have no duplicates and no specific order (unless it’s a SortedSet). When I see a Collection, I don’t expect anything more from it than to contain some entities.
Regarding list ordering… Yes, it can be preserved. And even if it’s not and you just use @OrderBy, it still can be useful. Think about the example of event log sorted by timestamp by default. Artificially reordering the list makes little sense, but still it can be useful that it comes sorted by default.