I have just recently run into this problem where assets were accidentally added to git-lfs on one branch that shouldn’t have been. My solution was:
git lfs untrack '<file-type>'
git rm --cached '<file-type>'
git add '<file-type>'
git commit -m "restore '<file-type>' to git from lfs"
The result is a rewrite of the git-lfs oid sha256 pointers with the standard file contents.
(Edit 2019-03): The accepted answer was changed to provide an easy solution for simpler cases. See also the edits in the answer by VonC for alternate solutions in case you have a more complex case on hand.