Is multithreading in python a myth?

Multithreading in Python is sort of a myth.

There’s technically nothing forbidding multiple threads from trying to access the same resource at the same time. The result is usually not desirable, so things like locks, mutexes, and resource managers were developed. They’re all different ways to ensure that only one thread can access a given resource at a time. In essence, they make threads play nice together. However, if a lot of the threads’ time is spent waiting for resources, you’re not getting any benefits from multithreading, and you’d be better off writing a single-threaded program instead (or restructuring your program to avoid the waiting).

That being said, in CPython (the most prevalent Python implementation – the one you get from clicking the download button on https://python.org or via a package manager), there’s this evil necessity called the Global Interpreter Lock (GIL). In order to make the dynamic memory management in CPython work correctly, the GIL prevents multiple threads from running Python code at the same time. This is because CPython’s dynamic memory management is not thread-safe – it can have those same problems of multiple threads accessing (or worse, disposing) the same resource at the same time. The GIL was a compromise between the two extremes of not allowing multi-threaded code, and having the dynamic memory management be very bulky and slow.

Other implementations (like Jython and IronPython, but not PyPy) don’t have a GIL, because the platforms they are built on (Java for Jython, .NET for IronPython) handle dynamic memory management differently, and so can safely run the Python code in multiple threads at the same time.

If you’re using CPython, it’s highly recommended to use the multiprocessing module instead. Rather than running multiple threads, it runs multiple processes (each with their own GIL, so they can all run at the same time). It’s much more effective than multithreading. The alternative is to write your multithreaded code in C/C++ as an extension, because native code is not subject to the GIL. However, that’s usually a lot more work, and the payoff is usually not worth the effort.


Regarding green threads: they don’t implement multithreading in the usual sense. Green threads are closer to coroutines, in that they (usually) can’t take advantage of multiple processor cores to run in true parallel. Instead, they typically implement cooperative multitasking, where each green thread will manually pass control to another green thread. Stackless Python has built-in support for green threads, and the greenlet extension brings them to CPython. There are probably other libraries/modules out there that implement green threads, but I’m not familiar with any others.

Leave a Comment

tech