In my case the SSH-authenticated repository was updated to use LFS from another client and on my side Git-LFS didn’t know about the SSH remote-url. What I did to fix it was the following:
Copy the URL configured in remote.origin.url (push URL for origin) to lfs.url (the URL LFS uses):
$ git config lfs.url $(git config remote.origin.url)
(If your remote is not named origin then change to your remote name.)
Then run
$ git config lfs.url
to show the URL and confirm that it does indeed contain an SSH url, and not some HTTP/HTTPS url.
Then you can
$ git pull
Done.
If you screwed up before and main and origin/main have somehow diverged as was the case for me then you might need to git checkout -fB main origin/main (this doesn’t ask and overwrites the local version of the main branch, so beware and execute carefully!).
See also: https://github.com/git-lfs/git-lfs/issues/2661#issuecomment-335903332