I figured it out, on this page of the documentation in the “BaseSerializer” section, there’s an example that shows ValidationError can take a dictionary argument upon initialization.
If I raise ValidationError({'field_val1': ['this field is not valid']}) I get the JSON response I want.