Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

More LEM benchmarks and Coproc derivied implementations #755

Merged
merged 1 commit into from
Oct 17, 2023

Conversation

arthurpaulino
Copy link
Member

[Post #738]

This PR implement the missing benchmarks for LEM. Here's the data I collected on my machine w.r.t. Alpha:

end2end proving

end2end_benchmark/end2end_go_base_nova/_10_0
                        time:   [1.1503 s 1.1551 s 1.1583 s]
                        change: [-19.754% -19.255% -18.666%] (p = 0.00 < 0.05)
                        Performance has improved.

synthesis

synthesis/Synthesis-rc/5
                        time:   [7.2469 ms 7.2700 ms 7.2888 ms]
                        change: [-64.198% -64.043% -63.892%] (p = 0.00 < 0.05)
                        Performance has improved.
synthesis/Synthesis-rc/10
                        time:   [14.649 ms 14.656 ms 14.664 ms]
                        change: [-63.761% -63.730% -63.699%] (p = 0.00 < 0.05)
                        Performance has improved.
synthesis/Synthesis-rc/100
                        time:   [151.00 ms 151.11 ms 151.21 ms]
                        change: [-63.398% -63.337% -63.283%] (p = 0.00 < 0.05)
                        Performance has improved.
synthesis/Synthesis-rc/200
                        time:   [298.47 ms 299.53 ms 300.16 ms]
                        change: [-63.713% -63.527% -63.346%] (p = 0.00 < 0.05)
                        Performance has improved.

@winston-h-zhang was correct: synthesis is ~2.75x faster in LEM

sha256 (IVC)

prove/2023-10-14:cff480d56f112511ebbfdba41799a31774492652:rc=10:sha256_ivc_1/1
                        time:   [7.7633 s 7.8085 s 7.8601 s]
                        change: [-31.809% -31.067% -30.408%] (p = 0.00 < 0.05)
                        Performance has improved.

I didn't have enough RAM for higher RCs

sha256 (NIVC)

prove/2023-10-14:cff480d56f112511ebbfdba41799a31774492652:rc=10:sha256_ivc_1/1
                        time:   [3.6714 s 3.6822 s 3.6908 s]
                        change: [-18,164% -18,566% -18,912%] (p = 0.00 < 0.05)
                        Performance has improved.

I didn't have enough RAM for higher RCs
Note: there's a typo in the group name, where it says "ivc"

fibonacci

Prove/2023-10-14:cff480d56f112511ebbfdba41799a31774492652:Fibonacci-rc=100/100
                        time:   [21.800 s 21.823 s 21.847 s]
                        change: [-19.070% -18.909% -18.743%] (p = 0.00 < 0.05)
                        Performance has improved.

I didn't have enough RAM for higher RCs

@arthurpaulino arthurpaulino force-pushed the more-lem-benchmarks branch 2 times, most recently from 1644a8d to 403e42d Compare October 16, 2023 13:27
@arthurpaulino arthurpaulino changed the title More LEM benchmarks More LEM benchmarks and Coproc derivied implementations Oct 16, 2023
@arthurpaulino arthurpaulino marked this pull request as ready for review October 16, 2023 23:46
@arthurpaulino arthurpaulino requested review from a team as code owners October 16, 2023 23:46
@arthurpaulino arthurpaulino mentioned this pull request Oct 17, 2023
huitseeker
huitseeker previously approved these changes Oct 17, 2023
Copy link
Contributor

@huitseeker huitseeker left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the ports!

@arthurpaulino arthurpaulino added this pull request to the merge queue Oct 17, 2023
Merged via the queue into master with commit e4f634c Oct 17, 2023
9 checks passed
@arthurpaulino arthurpaulino deleted the more-lem-benchmarks branch October 17, 2023 14:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants