TreeMap seems a no-brainer to me – simply because of the “in alphabetical order” requirement. HashMap has no ordering when you iterate through it; TreeMap iterates in the natural key order.
EDIT: I think Konrad’s comment may have been suggesting “use HashMap, then sort.” This is good because although we’ll have N iterations initially, we’ll have K <= N keys by the end due to duplicates. We might as well save the expensive bit (sorting) until the end when we’ve got fewer keys than take the small-but-non-constant hit of keeping it sorted as we go.
Having said that, I’m sticking to my answer for the moment: because it’s the simplest way of achieving the goal. We don’t really know that the OP is particularly worried about performance, but the question implies that he’s concerned about the elegance and brevity. Using a TreeMap makes this incredibly brief, which appeals to me. I suspect that if performance is really an issue, there may be a better way of attacking it than either TreeMap or HashMap 🙂