Support for custom allocators

[lquerel]

I’d like to propose adding features to support mimalloc and jemalloc. Regorus is typically a CPU and memory-bound system, so it makes sense to use allocators that are more efficient than the standard ones. I’ve seen significant performance gains in several of my projects simply by enabling this type of allocator. It seems worthwhile to support this, especially for the bindings, as it doesn’t appear that it can be easily done without modifying the binding code.

I can create a PR if the maintainers of this project are interested in the idea.

[anakrish]

@lquerel Thanks for the proposal. I agree that it will be in general be beneficial to support custom allocators. Please do create a PR.

[lquerel]

@anakrish I did a quick and dirty test with the standard allocator, mimalloc, and jemalloc (see the screenshot below, env: macos). Without any concurrency, the results were very close, with a slight advantage for mimalloc. However, after modifying the regorus CLI (examples) and running 100 tasks in parallel, the differences between the allocators became clearly apparent. This was expected, as mimalloc, in particular, is optimized for multithreaded environments.

I recommend always enabling mimalloc for the bindings since there is no disadvantage to using it in single-threaded scenarios, and there is a significant advantage in scenarios involving multiple policy evaluations in parallel, which I believe is a common use case in production. What do you think?

[anakrish]

@lquerel Thanks for performing the experiments. The performance benefits look great! Would it be possible to also try out https://github.com/schrodingerzhu/snmalloc-rs? It is an allocator that should perform well in multi-threaded scenarios.

I recommend always enabling mimalloc for the bindings since there is no disadvantage to using it in single-threaded scenarios, and there is a significant advantage in scenarios involving multiple policy evaluations in parallel, which I believe is a common use case in production. What do you think?

I concur. However, it would be great if the allocator was made configurable so that users can choose the allocator that they favor.

[lquerel]

@anakrish It looks like mimalloc and snmalloc are equivalent in terms of performance for this scenario.

I will create a PR later today with a set of cargo features.

[anakrish]

@lquerel Thanks!

[anakrish]

@lquerel You'd already have likely seen this:

regorus/bindings/ffi/src/lib.rs

Line 440 in edd0ccc

#[cfg(feature = "custom_allocator")]

This is mainly used for no_std environments and is currently not exposed to the different bindings.