Pure Python sucks in the scene of parallel computing, due to the existence of the Global Interpreter Lock (aka GIL). GIL prevents accessing or manipulating interpreter from different threads concurrently. The mechanism alleviates the risk of race condition, but sequentializes multi-threading program so that it behaves just like a single-threading version as well. Sadly, there’s no way to release the lock from pure Python.
Alright. So what about beyond pure Python? Shall we bypass the mechanism within an extension? The answer is yes, and that’s what most of scientific computing libaries do.
Cython is a good choice for writing extensions, less verbose, and more similar to Python syntactically. In Cython, one can release GIL temporarily for a code block using the
with nogil: syntax. Will it release the true power of multi-core CPU? We should have a try.
We adopt a toy example, say, a naive matrix multiplication, for benchmarking. Start with a C-only version:
The function above is straight-forward. We then create a wrapper for it, so that it can be called by Python code:
Now the Cython part is ready. Below a script for benchmarking:
Two matrices with a rather large size
1200 x 1200 are supplied as input, and we test
matmul against four settings. The result is listed as below:
The first two rows show that, with single thread,
matmul has comparable performance no matter releasing GIL or not. This is desired behavior, since GIL should not lead to performance degradation in single-threading scene. But things change when it comes to multi-threading. With two computing threads running in parallel, the time doubles if holding GIL, whilst in another setting (GIL released), the performance remains unchanged.
We may step further to investigate the behavior of
prange is provided by Cython for more convenient parallel computing, adopting the famous OpenMP as backend. Writing a
_matmul should take minor modification:
cdef void _matmul_p(
plus the wrapper
and also, the benchmark script:
for kw in make_grid(
OpenMP requires extra compilation flags, so a
.pyxbld file is needed:
|nthreads||GIL||time w/o par. (s)||time w/ par. (s)|
We can see that
prange brings an amazing boost in performance!
_matmul_p is 3~4x faster in single-threading setting. The number might vary across different hardwares, depending on the number of CPU cores. In the setting of two threads, the running time doubles, which indicates that
prange does efficiently use up all available CPU resources.
Cython supports native parallelism through the cython.parallel module. To use this kind of parallelism, the GIL must be released (see Releasing the GIL). It currently supports OpenMP, but later on more backends might be supported. – Using Parallelism
If there’s no need to hold GIL, just release it. This happens when you are manipulating some C data structures, and not attempting to disturb the interpreter.
If there’s massive looping in your Cython code, feel free to accelerate it with
prange will effeciently schedule the computation onto all CPU resources.
If there’s some macro tasks
routines consisting of large pieces of logic which could not be easily parallelized in Cython, schedule them via
threading sucks for most of the time, but if the tasks not always acquiring GIL, it should be fine just like threads in other languages.