^? is the delete character; the backspace character is ^H. Only one of these is recognized by your terminal as “erase”, and this is determined by the terminal settings, stty. (bash and other shells understand this as a problem and do special things to recognize both)
If your terminal emulator (ssh, putty, xterm, whatever) disagrees with your terminal settings, then you see this behavior. Usually it’s right by default, but very often people will put stty commands in their .bashrc which breaks things.
You probably have something like stty erase ^H in your bashrc. If you do, get rid of it, or change your terminal settings to have backspace send ^H instead of DEL (^?)
You can also fix this with vim mappings, but that’s ignoring the basic problem.