This worked for me, but with a whole directory.
As shown here
~$ cd neu
~/neu$ git filter-branch --subdirectory-filter FooBar HEAD
~/neu$ git reset --hard
~/neu$ git remote rm origin
~/neu$ rm -r .git/refs/original/
~/neu$ git reflog expire --expire=now --all
~/neu$ git gc --aggressive
~/neu$ git prune
~/neu$ git remote add origin git://github.com/FooBar/neu.git
EDIT: For a single file:
Filter the directory first:
git filter-branch --prune-empty --subdirectory-filter myDirectory -- --all
Filter the single file:
git filter-branch -f --prune-empty --index-filter "git rm --cached --ignore-unmatch $(git ls-files | grep -v 'keepthisfile.txt')"
Do some cleanup:
git reset --hard
git gc --aggressive
git prune
This should do it.