The repo contains the source code for the Making Python 100x faster with less than 100 lines of Rust blog post (there's a copy in this repo as well).
It's a demo library in Python that we can convert parts of to Rust, to improve its performance.
Here's a table summarizing the perf gains:
| Version | Avg time per iteration (ms) | Multiplier |
|---|---|---|
| v1 - Baseline implementation (Python) | 293.41 | 1x |
v2 - Naive line-to-line translation of find_close_polygons |
23.44 | 12.50x |
v3 - Polygon implementation in Rust |
6.29 | 46.53x |
| v4 - Optimized allocations | 2.90 | 101.16x |
| v5 - Move outer loop to Rust (*) | 0.81 | 362.23x |
(*) v5 was added after the post was published, showing how to reduce Python to Rust overhead even more.
There are also two versions which use "vectorizing" (doing more of the work directly in numpy). While not applicable to the original library, they also show major performance gains:
| Version | Avg time per iteration (ms) | Multiplier |
|---|---|---|
| v1 - Baseline implementation (Python) | 293.41 | 1x |
v1.5 - A bit of numpy vectorization |
41.79 | 7.02x |
v1.6 - A lot of numpy vectorization, by @dangrie158 |
5.18 | 56.68x |
The code should work on all supported platforms (Python & Rust),
but --native profiling is only supported by py-spy on x86 Linux and Windows.
(macOS / Arm Linux can still generate profiles viewing just the Python code.)
For the complete Rust setup, you'll need Rust (use rustup).
To setup with rye, run:
rye syncTo build the native extension, run:
source .venv/bin/activate
(cd poly_match_rs && maturin develop --release)To run the benchmarks, run:
rye run python ./measure_all.pyFor profiling,
sudo rye run py-spy record -o profile.svg -- python measure.pyThere are a few more optimizations we can try.
For example, we could use (dx^2 + dy^2) < dist^2 (calculating dist^2 only once, saving a sqrt).
According to the profiler outputs, can we expect this to make a big difference?
We could build a Rust bench using criterion,
which would require us to "split" the Python & pure-Rust parts (Maybe using AsRef<Polygon>).
We could also try to convert the list of Polygons only once to Rust at the start of the run.
According to the profiler outputs, can we expect this to make a big difference? (See v5)
This repo is licensed under either of
Apache License, Version 2.0, (LICENSE-APACHE or http://www.apache.org/licenses/LICENSE-2.0)
MIT license (LICENSE-MIT or http://opensource.org/licenses/MIT)
at your option.