You can disable this by running git config commit.gpgsign false This sets the configuration locally instead of globally.
Putting this setting in .gitconfig worked for me with what you had, without the [user] configuration:
[commit]
gpgsign = false