If the permissions are correct, SELinux might still be preventing sshd from opening the file.
Try fixing the labels inside the .ssh directory (and maybe $HOME):
restorecon -FRvv ~/.ssh
If the user account uses non-standard home path, default labels for the path need to be added to the local configuration first:
semanage fcontext -a -t ssh_home_t "/srv/custom/\.ssh(/.*)?"
(I’m intentionally not suggesting disabling SELinux or setting it to the permissive mode.)