Force exceptions language in English

You could set the current culture to English only in debug builds :

#if DEBUG
Thread.CurrentThread.CurrentUICulture = new CultureInfo("en-US");
#endif