You can create rather complex regular expression but it isn’t worth that. Using exceptions here is absolutely normal.
It’s natural exceptional situation: you assume that there is an integer in the string but indeed there is something else. Exception should be thrown and handled properly.
If you look inside parseLong code, you’ll see that there are many different verifications and operations. If you want to do all that stuff before parsing it’ll decrease the performance (if we are talking about parsing millions of numbers because otherwise it doesn’t matter). So, the only thing you can do if you really need to improve performance by avoiding exceptions is: copy parseLong implementation to your own function and return NaN instead of throwing exceptions in all correspondent cases.