With a solution open, go to the Debug – Windows – Exception Settings (Ctrl+Alt+E) menu option. From there you can choose to break on Thrown or User-unhandled exceptions.
EDIT: My instance is set up with the C# “profile” perhaps it isn’t there for other profiles?