If you’re not getting prompted at all for a passphrase, the solution may just be to install a program to facilitate that. The most common is pinentry.
brew install pinentry-mac
So installing that and trying again may get things working. But if not, another thing to do is make sure git it using/finding the right GPG program. These days you really should be using gpg2, so if you don’t already have that installed, do this:
gpg --version
…and make sure it indicates you have GnuPG version 2+ (not version 1) installed.
If you already have GnuPG 2+ and pinentry installed, then try this:
echo "pinentry-program /usr/local/bin/pinentry-mac" >> ~/.gnupg/gpg-agent.conf
…or, more robustly:
echo "pinentry-program $(which pinentry-mac)" >> ~/.gnupg/gpg-agent.conf
…and then try again.
And you may also need to stop gpg-agent:
gpgconf --kill gpg-agent
You don’t need to manually restart it — it will get restarted automatically when it’s needed.
Note: Some commenters mention needing to reboot after making changes — but it seems likely the only effect of that is to cause gpg-agent to be restarted. So manually killing gpg-agent as described above should be sufficient.