phoneNumber.replaceAll("[\\s\\-()]", "");
The regular expression defines a character class consisting of any whitespace character (\s, which is escaped as \\s because we’re passing in a String), a dash (escaped because a dash means something special in the context of character classes), and parentheses.
See String.replaceAll(String, String).
EDIT
Per gunslinger47:
phoneNumber.replaceAll("\\D", "");
Replaces any non-digit with an empty string.