Use an EditTextPreference and set the input type to TYPE_CLASS_NUMBER. This will force the user to enter numbers and not letters.
EditTextPreference pref = (EditTextPreference)findPreference("preference_name");
pref.getEditText().setInputType(InputType.TYPE_CLASS_NUMBER);