How to dynamically create a derived type in the Python C-API

I encountered the same problem when I was modifying an extension to be compatible with Python 3, and found this page when I was trying to solve it.

I did eventually solve it by reading the source code for the Python interpreter, PEP 0384 and the documentation for the C-API.

Setting the Py_TPFLAGS_HEAPTYPE flag tells the interpreter to recast your PyTypeObject as PyHeapTypeObject, which contains additional members that must also be allocated. At some point the interpreter attempts to refer to these extra members and, if you leave them unallocated, it will cause a segfault.

Python 3.2 introduced the C structures PyType_Slot and PyType_Spec and the C function PyType_FromSpec that simplify the creation of dynamic types. In a nutshell, you use PyType_Slot and PyType_Spec to specify the tp_* members of the PyTypeObject and then call PyType_FromSpec to do the dirty work of allocating and initialising the memory.

From PEP 0384, we have:

typedef struct{
  int slot;    /* slot id, see below */
  void *pfunc; /* function pointer */
} PyType_Slot;

typedef struct{
  const char* name;
  int basicsize;
  int itemsize;
  int flags;
  PyType_Slot *slots; /* terminated by slot==0. */
} PyType_Spec;

PyObject* PyType_FromSpec(PyType_Spec*);

(The above isn’t a literal copy from PEP 0384, which also includes const char *doc as a member of PyType_Spec. But that member doesn’t appear in the source code.)

To use these in the original example, assume we have a C structure, BrownNoddy, that extends the C structure for the base class Noddy. Then we would have:

PyType_Slot slots[] = {
    { Py_tp_doc, "BrownNoddy objects" },
    { Py_tp_base, &NoddyType },
    { Py_tp_new, BrownNoddy_new },
    { 0 },
};
PyType_Spec spec = { "noddy.BrownNoddy", sizeof(BrownNoddy), 0,
                      Py_TPFLAGS_DEFAULT | Py_TPFLAGS_BASETYPE, slots };
PyTypeObject *BrownNoddyType = (PyTypeObject *)PyType_FromSpec(&spec);

This should do everything in the original code, including calling PyType_Ready, plus what is necessary for creating a dynamic type, including setting Py_TPFLAGS_HEAPTYPE, and allocating and initialising the extra memory for a PyHeapTypeObject.

I hope that’s helpful.