Regex to remove letters, symbols except numbers

You can use \D which means non digits.

var removedText = self.val().replace(/\D+/g, '');

jsFiddle.

You could also use the HTML5 number input.

<input type="number" name="digit" />

jsFiddle.