It is a server error, not a client error. If server errors weren’t to be returned to the client, there wouldn’t have been created an entire status code class for them (i.e. 5xx).
You can’t hide the fact that you either made a programming error or some service you rely on is unavailable, and that certainly isn’t the client’s fault. Returning any other range of code in those cases than the 5xx series would make no sense.
RFC 7231 mentions in section 6.6. Server Error 5xx:
The 5xx (Server Error) class of status code indicates that the server
is aware that it has erred or is incapable of performing the
requested method.
This is exactly the case. There’s nothing “internal” about the code “500 Internal Server Error” in the sense that it shouldn’t be exposed to the client.