This is an ssh issue, not a git issue. What you are seeing is not a git whine, it’s a valid warning from your ssh client. You should ensure that your ssh config doesn’t attempt X11 forwarding by default (git doesn’t need it), or at least turn it off for the host you are connecting to.
In your ~/.ssh/config try:
ForwardX11 no
either in the host specific section or globally.