Welcome to the world of Abominable Function Types.
void() & is not a reference to void(). The way to spell that would be void(&)() (which if you remove_reference_t, you would get back void() — that is remove_reference_t does work on references to functions, if what you provide it is actually a reference to function type).
What void() & actually refers to is the type of a reference-qualified member function after you strip off the class. That is:
struct C {
void f() &;
};
The type of &C::f is void (C::*)() &. But all pointers to members can be written as T C::* for some type T, and in this case the type T would be void() &.
See also P0172.