Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Better batch commit and switch to Reed Solomon code. #155

Merged
merged 68 commits into from
Sep 9, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
68 commits
Select commit Hold shift + click to select a range
ff657db
Add basefold to master.
yczhangsjtu Aug 13, 2024
398949c
Refactor.
yczhangsjtu Aug 15, 2024
688cfc9
Fixes.
yczhangsjtu Aug 19, 2024
440360c
Profile speed.
yczhangsjtu Aug 19, 2024
289b783
API change to allow more polys in one commit.
yczhangsjtu Aug 19, 2024
42886af
Fix API change.
yczhangsjtu Aug 19, 2024
2126194
Implement simple batch protocol (untested)
yczhangsjtu Aug 20, 2024
abe1baf
Remove unnecessary features.
yczhangsjtu Aug 21, 2024
cfa0ce9
Move expensive tests to benchmarks.
yczhangsjtu Aug 21, 2024
1a44ea1
Merge branch 'fix-basefold' into basefold-improve-open
yczhangsjtu Aug 21, 2024
d372af9
Fix bugs introduced in batching Merkle tree leaves.
yczhangsjtu Aug 21, 2024
63c66d6
Make sumcheck test cheap.
yczhangsjtu Aug 21, 2024
c1b8488
Remove unnecessary timers that cause failure in parallel.
yczhangsjtu Aug 21, 2024
652e18b
Remove `print-trace` feature.
yczhangsjtu Aug 21, 2024
68c9ac0
Merge branch 'fix-basefold' into basefold-improve-open
yczhangsjtu Aug 22, 2024
27b8afb
Remove batch commit and write in test.
yczhangsjtu Aug 22, 2024
421b6eb
Use criterion in benchmark.
yczhangsjtu Aug 22, 2024
5876afe
Update commit_open_verify.rs
yczhangsjtu Aug 22, 2024
ac5e9e3
Merge branch 'master' into fix-basefold
yczhangsjtu Aug 22, 2024
dd846f2
Update Cargo.toml
yczhangsjtu Aug 22, 2024
6ad715f
basefold batch commit (#157)
iammadab Aug 22, 2024
a26dafa
Use criterion for benchmark.
yczhangsjtu Aug 22, 2024
ca820ca
WIP
yczhangsjtu Aug 22, 2024
20b708c
Fix bug.
yczhangsjtu Aug 22, 2024
4ee039b
Update test code.
yczhangsjtu Aug 22, 2024
7f23cd6
Provide polys with the same size for simple batch.
yczhangsjtu Aug 22, 2024
0fb02ea
Merge branch 'fix-basefold' into basefold-improve-open
yczhangsjtu Aug 22, 2024
829c90a
Split the benchmarks.
yczhangsjtu Aug 22, 2024
5b32b98
Merge branch 'fix-basefold' into basefold-improve-open
yczhangsjtu Aug 22, 2024
bd12f59
Add benchmark for simple batch.
yczhangsjtu Aug 22, 2024
18abd4b
Fix benchmark id.
yczhangsjtu Aug 22, 2024
f0bb755
Update commit_open_verify.rs
yczhangsjtu Aug 22, 2024
e05fd3e
Fix some issues in benchmark.
yczhangsjtu Aug 23, 2024
9f56dc7
Merge remote-tracking branch 'origin/master' into basefold-improve-open
yczhangsjtu Aug 23, 2024
9dcda7f
Merge remote-tracking branch 'origin/master' into basefold-improve-open
yczhangsjtu Aug 26, 2024
6fd2bbb
Fix batch open benchmark.
yczhangsjtu Aug 26, 2024
9374976
Refactor: put encoding in a trait.
yczhangsjtu Aug 27, 2024
65ea057
Ensure consistent weight for prover and verifier.
yczhangsjtu Aug 27, 2024
f9b4ecd
WIP: added FFT code from plonky2.
yczhangsjtu Aug 27, 2024
3360d89
Add benchmark for encoding.
yczhangsjtu Aug 27, 2024
39b9632
Implement RS code.
yczhangsjtu Aug 27, 2024
c1d1852
Fix.
yczhangsjtu Aug 29, 2024
0f153c5
Naive fft and fft comparison test passed.
yczhangsjtu Aug 30, 2024
d641ae4
Fix.
yczhangsjtu Aug 30, 2024
d79b184
Folding coefficients consistency passed.
yczhangsjtu Aug 30, 2024
4131dd0
Some fixes and benchmark.
yczhangsjtu Sep 2, 2024
1dbf6cc
Basecode folding test passed.
yczhangsjtu Sep 2, 2024
11c9281
Allow config message bitreversion.
yczhangsjtu Sep 2, 2024
f375893
Add another two tests.
yczhangsjtu Sep 2, 2024
139dbf5
Add test_low_degree.
yczhangsjtu Sep 2, 2024
cbc6512
Check low degree of special foldings.
yczhangsjtu Sep 2, 2024
06db748
Folding test passed.
yczhangsjtu Sep 2, 2024
8c380dc
Use real challenge in test.
yczhangsjtu Sep 2, 2024
03607a8
Test continued folding.
yczhangsjtu Sep 2, 2024
74d1f76
Test rscode and basecode separately.
yczhangsjtu Sep 2, 2024
0ecbd55
Update the logic of folding message.
yczhangsjtu Sep 2, 2024
9f6d03a
Conditional reverse bits of messages.
yczhangsjtu Sep 2, 2024
ada0c97
Revert "Update the logic of folding message."
yczhangsjtu Sep 2, 2024
9f9964b
Revert "Conditional reverse bits of messages."
yczhangsjtu Sep 2, 2024
1d82beb
All tests passed.
yczhangsjtu Sep 2, 2024
d7084d7
Rename to avoid confusion.
yczhangsjtu Sep 2, 2024
5154975
More benchmarks.
yczhangsjtu Sep 2, 2024
d6e75ec
Increase number of queries for 100 security bits.
yczhangsjtu Sep 3, 2024
3d59141
Fix clippy warning, make code cleaner
dreamATD Sep 4, 2024
702a1de
Merge remote-tracking branch 'origin/master' into basefold-improve-open
yczhangsjtu Sep 5, 2024
114ec36
Merge remote-tracking branch 'origin/master' into basefold-improve-open
yczhangsjtu Sep 6, 2024
77d4078
Merge remote-tracking branch 'origin/master' into basefold-improve-open
yczhangsjtu Sep 9, 2024
0f3cf90
Add a FIXME.
yczhangsjtu Sep 9, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 21 additions & 1 deletion mpcs/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -46,5 +46,25 @@ sanity-check = []
print-trace = [ "ark-std/print-trace" ]

[[bench]]
name = "commit_open_verify"
name = "commit_open_verify_rs"
harness = false

[[bench]]
name = "commit_open_verify_basecode"
harness = false

[[bench]]
name = "basecode"
harness = false

[[bench]]
name = "rscode"
harness = false

[[bench]]
name = "interpolate"
harness = false

[[bench]]
name = "fft"
harness = false
110 changes: 110 additions & 0 deletions mpcs/benches/basecode.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,110 @@
use std::time::Duration;

use criterion::*;
use ff::Field;
use goldilocks::GoldilocksExt2;

use itertools::Itertools;
use mpcs::{
util::{
arithmetic::interpolate_field_type_over_boolean_hypercube,
plonky2_util::reverse_index_bits_in_place_field_type,
}, Basefold, BasefoldBasecodeParams, BasefoldSpec, EncodingScheme, PolynomialCommitmentScheme
};

use multilinear_extensions::mle::{DenseMultilinearExtension, FieldType};
use rand::{rngs::OsRng, SeedableRng};
use rand_chacha::ChaCha8Rng;
use rayon::iter::{IntoParallelRefIterator, ParallelIterator};

type Pcs = Basefold<GoldilocksExt2, BasefoldBasecodeParams, ChaCha8Rng>;
type E = GoldilocksExt2;

const NUM_SAMPLES: usize = 10;
const NUM_VARS_START: usize = 15;
const NUM_VARS_END: usize = 20;
const BATCH_SIZE_LOG_START: usize = 3;
const BATCH_SIZE_LOG_END: usize = 5;

fn bench_encoding(c: &mut Criterion, is_base: bool) {
let mut group = c.benchmark_group(format!(
"encoding_basecode_{}",
if is_base { "base" } else { "extension" }
));
group.sample_size(NUM_SAMPLES);
// Challenge is over extension field, poly over the base field
for num_vars in NUM_VARS_START..=NUM_VARS_END {
for batch_size_log in BATCH_SIZE_LOG_START..=BATCH_SIZE_LOG_END {
let batch_size = 1 << batch_size_log;
let rng = ChaCha8Rng::from_seed([0u8; 32]);
let (pp, _) = {
let poly_size = 1 << num_vars;
let param = Pcs::setup(poly_size, &rng).unwrap();
Pcs::trim(&param, poly_size).unwrap()
};
let polys = (0..batch_size)
.map(|_| {
if is_base {
DenseMultilinearExtension::random(num_vars, &mut rng.clone())
} else {
DenseMultilinearExtension::from_evaluations_ext_vec(
num_vars,
(0..1 << num_vars).map(|_| E::random(&mut OsRng)).collect(),
)
}
})
.collect_vec();


group.bench_function(
BenchmarkId::new("batch_encode", format!("{}-{}", num_vars, batch_size)),
|b| {
b.iter(|| {
polys
.par_iter()
.map(|poly| {
// Switch to coefficient form
let mut coeffs = poly.evaluations.clone();
interpolate_field_type_over_boolean_hypercube(&mut coeffs);

let mut codeword =
<<BasefoldBasecodeParams as BasefoldSpec<E>>::EncodingScheme as EncodingScheme<E>>::encode(
&pp.encoding_params,
&coeffs,
);

// If using repetition code as basecode, it may be faster to use the following line of code to create the commitment and comment out the two lines above
// let mut codeword = evaluate_over_foldable_domain(pp.log_rate, coeffs, &pp.table);

// The sum-check protocol starts from the first variable, but the FRI part
// will eventually produce the evaluation at (alpha_k, ..., alpha_1), so apply
// the bit-reversion to reverse the variable indices of the polynomial.
// In short: store the poly and codeword in big endian
reverse_index_bits_in_place_field_type(&mut coeffs);
reverse_index_bits_in_place_field_type(&mut codeword);

(coeffs, codeword)
})
.collect::<(Vec<FieldType<E>>, Vec<FieldType<E>>)>();
})
},
);
}
}
}

fn bench_encoding_goldilocks_2(c: &mut Criterion) {
bench_encoding(c, false);
}

fn bench_encoding_base(c: &mut Criterion) {
bench_encoding(c, true);
}

criterion_group! {
name = bench_basefold;
config = Criterion::default().warm_up_time(Duration::from_millis(3000));
targets = bench_encoding_base, bench_encoding_goldilocks_2,
}

criterion_main!(bench_basefold);
Loading
Loading