feat: add new parallel implementation for permute_expression_pair

[jonathanpwang]

A new algorithm to get (A', S') that is fully multi-threaded: this is a different algorithm than the original permute_expression_pair_seq.

I observed that the previous computation was still single-threaded at some places, which becomes a bottleneck for larger circuits. This is because the way A', S' need to be permuted is rather esoteric and not so parallel friendly.

My implementation isn't optimized by any means: I just aggressively use rayon and fold on BTreeMap (in Axiom's repo I used HashMap so I'm not sure of the performance difference with BTreeMap).

Also I'm not sure why indexing into Range wasn't implement on Polynomial before but it was on RangeTo. I could also add it for RangeFrom if there's interest.

[CPerezz]

Still need to analyze the algorithm in depth. So far looks well. Will give it another look later today.

Also, left some comments. And wanted to mention that it would be nice to see at least some numbers in regards performance changes/memory consumption changes so that we know whether this is feasible to be merged.

[CPerezz]

If we're already checking for empty range, then we should unwrap

Suggested change

	let prev_end = input_ranges.last().unwrap().1.end;
	let prev_end = unsafe{ input_ranges.last().unwrap_unchecked().1.end};

[CPerezz]

Not sure how we know we will never panic here.

[jonathanpwang]

Each r1 is the result of the previous fold step. As long as the fold is over nonempty iterator, the output should be nonempty. So r1 is nonempty unless input_uniques is empty I believe.

[CPerezz]

Hmmmm We can maybe file an issue in case we see this being critical.

[CPerezz]

Also I'm not sure why indexing into Range wasn't implement on Polynomial before but it was on RangeTo. I could also add it for RangeFrom if there's interest.

I'm planning to write a polynomial lib for ff/group-based libs. Something similar to ark-poly in arkworks. And not only for univariate but for multivariate polynomials too..
Will it be interesting to you guys? @jonathanpwang

[jonathanpwang]

Still need to analyze the algorithm in depth. So far looks well. Will give it another look later today.

Also, left some comments. And wanted to mention that it would be nice to see at least some numbers in regards performance changes/memory consumption changes so that we know whether this is feasible to be merged.

Sure, what kind of machine do you usually bench on? Not sure my Macbook will be a good standard heh.

I'm planning to write a polynomial lib for ff/group-based libs. Something similar to ark-poly in arkworks. And not only for univariate but for multivariate polynomials too..

Sounds nice! Right now I don't have to use polynomial stuff much outside of just halo2_proofs, but having all the traits so you can use them as slices would be very helpful. Things like this indexing stuff, AsRef<[F]> and Deref<[F]> etc.

[CPerezz]

Sure, what kind of machine do you usually bench on? Not sure my Macbook will be a good standard heh.

I bench on my laptop too. It has 16CPUs. So enough to see if it's indeed better performance-wise.
I'd say with your mac it should be fine to see the general improvement tendencies. Otherwise @ed255 might know how we can bench that in our servers.

[ed255]

Sure, what kind of machine do you usually bench on? Not sure my Macbook will be a good standard heh.

I bench on my laptop too. It has 16CPUs. So enough to see if it's indeed better performance-wise. I'd say with your mac it should be fine to see the general improvement tendencies. Otherwise @ed255 might know how we can bench that in our servers.

We have done some end-to-end benchmarking on AWS servers with number of cores between 16 and 128. We haven't really studied the bench results with high number of cores, so I think for this PR benchmarks with 8 or 16 cores should be good enough

[CPerezz]

@jonathanpwang Did you collect some numbers on your Mac?

[CPerezz]

ping @jonathanpwang

[jonathanpwang]

Sorry, was busy. It was also a bit hard because there was no existing lookup benchmarking using a real prover. I originally wanted to test just the permute_expression_pair function, but due to crate privacy issues this wasn't possible.

I added a very basic lookup bench, which is not very comprehensive since the lookups are very uniform. It uses IPA, but I don't think that matters for benchmarking lookup permutation.

For now, the benchmarks on the bench_lookup I added are in this commit message: e0b4479

On my laptop (M2Max), it seems the parallelized version isn't any better, but I think this is because the circuit I used is too simple. I'd like to do the benchmarks on the zkevm keccak circuit, but I need to resolve some versioning issues for that first. I will post an update once I have those benchmarks.

[alexander-camuto]

@jonathanpwang @CPerezz We have some pretty large circuits that we could benchmark on if that helps. Would need to update this branch with #192 once merged for it to work -- then can send some numbers yonder