The shortest way I’ve come up with is two statements:
assertThat( affA.entrySet(), everyItem(isIn(affB.entrySet())));
assertThat( affB.entrySet(), everyItem(isIn(affA.entrySet())));
But you can probably also do:
assertThat(affA.entrySet(), equalTo(affB.entrySet()));
depending on the implementations of the maps, and sacrificing the clarity of the difference report: that would just tell you that there is a difference, while the statement above would also tell you which one.
UPDATE: actually there is one statement that works independently of the collection types:
assertThat(affA.entrySet(), both(everyItem(isIn(affB.entrySet()))).and(containsInAnyOrder(affB.entrySet())));