How to make json.dumps in Python ignore a non-serializable field

Question

Keys with a leading _ underscore are not really ‘hidden’, they are just more strings to JSON. The Construct Container class is just a dictionary with ordering, the _io key is not anything special to that class.

You have two options:

implement a default hook that just returns a replacement value.
Filter out the key-value pairs that you know can’t work before serialising.

and perhaps a third, but a casual scan of the Construct project pages doesn’t tell me if it is available: have Construct output JSON or at least a JSON-compatible dictionary, perhaps by using adapters.

The default hook can’t prevent the _io key from being added to the output, but would let you at least avoid the error:

json.dumps(packet, default=lambda o: '<not serializable>')

Filtering can be done recursively; the @functools.singledispatch() decorator can help keep such code clean:

from functools import singledispatch

_cant_serialize = object()

@singledispatch
def json_serializable(object, skip_underscore=False):
    """Filter a Python object to only include serializable object types

    In dictionaries, keys are converted to strings; if skip_underscore is true
    then keys starting with an underscore ("_") are skipped.

    """
    # default handler, called for anything without a specific
    # type registration.
    return _cant_serialize

@json_serializable.register(dict)
def _handle_dict(d, skip_underscore=False):
    converted = ((str(k), json_serializable(v, skip_underscore))
                 for k, v in d.items())
    if skip_underscore:
        converted = ((k, v) for k, v in converted if k[:1] != '_')
    return {k: v for k, v in converted if v is not _cant_serialize}

@json_serializable.register(list)
@json_serializable.register(tuple)
def _handle_sequence(seq, skip_underscore=False):
    converted = (json_serializable(v, skip_underscore) for v in seq)
    return [v for v in converted if v is not _cant_serialize]

@json_serializable.register(int)
@json_serializable.register(float)
@json_serializable.register(str)
@json_serializable.register(bool)  # redudant, supported as int subclass
@json_serializable.register(type(None))
def _handle_default_scalar_types(value, skip_underscore=False):
    return value

I have the above implementation an additional skip_underscore argument too, to explicitly skip keys that have a _ character at the start. This would help skip all additional ‘hidden’ attributes the Construct library is using.

Since Container is a dict subclass, the above code will automatically handle instances such as packet.

Leave a Comment Cancel reply