Pure Python sucks in the scene of parallel computing, due to the existence of the Global Interpreter Lock (aka GIL). GIL prevents accessing or manipulating interpreter from different threads concurrently. The mechanism alleviates the risk of race condition, but sequentializes multi-threading program as well. Sadly, there’s no way to release the lock from pure Python.

Alright. So what about beyond pure Python? Shall we bypass the mechanism within an extension? The answer is yes, and that’s what most of scientific computing libaries do.

Cython is a good choice for writing extensions, less verbose, and more similar to Python syntactically. In Cython, one can release GIL temporarily for a code block using the with nogil: syntax. Will it release the true power of multi-core CPU? We should have a try.

We adopt a toy example, say, a naive matrix multiplication, for benchmarking. Start with a C-only version:

The function above is straight-forward. We then create a wrapper for it, so that it can be called by Python code:

Now the Cython part is ready. Below a script for benchmarking:

Two matrices with a rather large size 1200 x 1200 are supplied as input, and we test matmul against four settings. The result is listed as below:

1 N 3.47
1 Y 3.51
2 N 3.78
2 Y 6.96

The first two rows show that, with single thread, matmul has comparable performance no matter releasing GIL or not. This is desired behavior, since GIL should not lead to performance degradation in single-threading scene. But things change when it comes to multi-threading. With two computing threads running in parallel, the time doubles if holding GIL, whilst in another setting (GIL released), the performance remains unchanged.

We may step further to investigate the behavior of prange. prange is provided by Cython for more convenient parallel computing, adopting the famous OpenMP as backend. Writing a prange version _matmul should take minor modification:

plus the wrapper matmul:

and also, the benchmark script:

OpenMP requires extra compilation flags, so a .pyxbld file is needed:

nthreads GIL time w/o par. (s) time w/ par. (s)
1 N 3.47 0.82
1 Y 3.51 0.89
2 N 3.78 1.79
2 Y 6.96 1.81

We can see that prange brings an amazing boost in performance! _matmul_p is 3~4x faster in single-threading setting. The number might vary across different hardwares, depending on the number of CPU cores. In the setting of two threads, the running time doubles, which indicates that prange does efficiently use up all available CPU resources.

We can also notice that, whether to release GIL or not seemingly does not affect prange . The reason is prange requires GIL to be released, which is automatically done by default.

Cython supports native parallelism through the cython.parallel module. To use this kind of parallelism, the GIL must be released (see Releasing the GIL). It currently supports OpenMP, but later on more backends might be supported. – Using Parallelism

## Conclusion

If there’s no need to hold GIL, just release it. This happens when you are manipulating some C data structures, and not attempting to disturb the interpreter.

If there’s massive looping in your Cython code, feel free to accelerate it with prange. prange will effeciently schedule the computation onto all CPU resources.

If there’s some macro tasks which could not be easily parallelized in Cython, schedule them via threading module. threading sucks for most of the time, but if the tasks not always acquiring GIL, it should be fine just like threads in other languages.