Setting the message of a custom Exception without passing it to the base constructor

Just put the formatting code into a static method?

public CustomException(dynamic json) : base(HumanReadable(json)) {}
private static string HumanReadable(dynamic json) {
    return whatever you need to;
}