Tracking PR for v0.7.0 release

.github/dependabot.yml

@@ -0,0 +1,6 @@
+version: 2
+updates:
+  - package-ecosystem: "cargo"
+    directory: "/"
+    schedule:
+      interval: "weekly"

CHANGELOG.md

@@ -1,5 +1,37 @@
 # Changelog
 
+## 0.7.0 (2023-10-11)
+
+#### Assembly
+- Added ability to attach doc comments to re-exported procedures (#994).
+- Added support for nested modules (#992).
+- Added support for the arithmetic expressions in constant values (#1026).
+- Added support for module aliases (#1037).
+- Added `adv.insert_hperm` decorator (#1042).
+- Added `adv.push_smtpeek` decorator (#1056).
+- Added `debug` decorator (#1069).
+- Refactored `push` instruction so now it parses long hex string in little-endian (#1076).
+
+#### CLI
+- Implemented ability to output compiled `.masb` files to disk (#1102).
+
+#### VM Internals
+- Simplified range checker and removed 1 main and 1 auxiliary trace column (#949).
+- Migrated range checker lookups to use LogUp and reduced the number of trace columns to 2 main and 
+  1 auxiliary (#1027).
+- Added `get_mapped_values()` and `get_store_subset()` methods to the `AdviceProvider` trait (#987).
+- [BREAKING] Added options to specify maximum number of cycles and expected number of cycles for a program (#998).
+- Improved handling of invalid/incomplete parameters in `StackOutputs` constructors (#1010).
+- Allowed the assembler to produce programs with "phantom" calls (#1019).
+- Added `TraceLenSummary` struct which holds information about traces lengths to the `ExecutionTrace` (#1029).
+- Imposed the 2^32 limit for the memory addresses used in the memory chiplet (#1049). 
+- Supported `PartialMerkleTree` as a secret input in `.input` file (#1072).
+- [BREAKING] Refactored `AdviceProvider` interface into [Host] interface (#1082).
+
+#### Stdlib
+- Completed `std::collections::smt` module by implementing `insert` and `set` procedures (#1036, #1038, #1046).
+- Added new module `std::crypto::dsa::rpo_falcon512` to support Falcon signature verification (#1000, #1094)
+
 ## 0.6.1 (2023-06-29)
 
 - Fixed `no-std` compilation for `miden-core`, `miden-assembly`, and `miden-processor` crates.

Makefile

@@ -1,5 +1,6 @@
 FEATURES_INTERNALS=--features internals
 FEATURES_CONCURRENT_EXEC=--features concurrent,executable
+FEATURES_GRAVITON_EXEC=--features concurrent,executable,sve
 FEATURES_METAL_EXEC=--features concurrent,executable,metal
 PROFILE_OPTIMIZED=--profile optimized
 PROFILE_TEST=--profile test-release
@@ -13,5 +14,8 @@ exec:
 exec-metal:
 	cargo build $(PROFILE_OPTIMIZED) $(FEATURES_METAL_EXEC)
 
+exec-graviton:
+	RUSTFLAGS="-C target-cpu=native" cargo build $(PROFILE_OPTIMIZED) $(FEATURES_GRAVITON_EXEC)
+
 test:
 	cargo test $(PROFILE_TEST) $(FEATURES_INTERNALS)

README.md

@@ -16,7 +16,7 @@ Miden VM is a zero-knowledge virtual machine written in Rust. For any program ex
 * If you'd like to learn more about STARKs, check out the [references](#references) section.
 
 ### Status and features
-Miden VM is currently on release v0.6. In this release, most of the core features of the VM have been stabilized, and most of the STARK proof generation has been implemented. While we expect to keep making changes to the VM internals, the external interfaces should remain relatively stable, and we will do our best to minimize the amount of breaking changes going forward.
+Miden VM is currently on release v0.7. In this release, most of the core features of the VM have been stabilized, and most of the STARK proof generation has been implemented. While we expect to keep making changes to the VM internals, the external interfaces should remain relatively stable, and we will do our best to minimize the amount of breaking changes going forward.
 
 The next version of the VM is being developed in the [next](https://github.com/0xPolygonMiden/miden-vm/tree/next) branch. There is also a documentation for the latest features and changes in the next branch [documentation next branch](https://0xpolygonmiden.github.io/miden-vm/intro/main.html).
 
@@ -31,7 +31,7 @@ Miden VM is a fully-featured virtual machine. Despite being optimized for zero-k
 * **Cryptographic operations.** Miden assembly provides built-in instructions for computing hashes and verifying Merkle paths. These instructions use the Rescue Prime Optimized hash function (which is the native hash function of the VM).
 * **External libraries.** Miden VM supports compiling programs against pre-defined libraries. The VM ships with one such library: Miden `stdlib` which adds support for such things as 64-bit unsigned integers. Developers can build other similar libraries to extend the VM's functionality in ways which fit their use cases.
 * **Nondeterminism**. Unlike traditional virtual machines, Miden VM supports nondeterministic programming. This means a prover may do additional work outside of the VM and then provide execution *hints* to the VM. These hints can be used to dramatically speed up certain types of computations, as well as to supply secret inputs to the VM.
-* **Custom advice providers.** Miden VM can be instantiated with user-defined advice providers. These advice providers are used to supply external data to the VM during execution/proof generation (via nondeterministic inputs) and can connect the VM to arbitrary data sources (e.g., a database or RPC calls).
+* **Customizable hosts.** Miden VM can be instantiated with user-defined hosts. These hosts are used to supply external data to the VM during execution/proof generation (via nondeterministic inputs) and can connect the VM to arbitrary data sources (e.g., a database or RPC calls).
 
 #### Planned features
 In the coming months we plan to finalize the design of the VM and implement support for the following features:
@@ -78,41 +78,58 @@ A few general notes on performance:
 * Both proof generation and proof verification times are greatly influenced by the hash function used in the STARK protocol. In the benchmarks below, we use BLAKE3, which is a really fast hash function.
 
 ### Single-core prover performance
-When executed on a single CPU core, the current version of Miden VM operates at around 10 - 15 KHz. In the benchmarks below, the VM executes a [Fibonacci calculator](miden/README.md#fibonacci-calculator) program on Apple M1 Pro CPU in a single thread. The generated proofs have a target security level of 96 bits.
+When executed on a single CPU core, the current version of Miden VM operates at around 20 - 25 KHz. In the benchmarks below, the VM executes a [Fibonacci calculator](miden/README.md#fibonacci-calculator) program on Apple M1 Pro CPU in a single thread. The generated proofs have a target security level of 96 bits.
 
 | VM cycles       | Execution time | Proving time | RAM consumed  | Proof size |
 | :-------------: | :------------: | :----------: | :-----------: | :--------: |
-| 2<sup>10</sup>  |  1 ms          | 80 ms        | 20 MB         | 47 KB      |
-| 2<sup>12</sup>  |  2 ms          | 260 ms       | 52 MB         | 57 KB      |
-| 2<sup>14</sup>  |  8 ms          | 0.9 sec      | 240 MB        | 66 KB      |
-| 2<sup>16</sup>  |  28 ms         | 4.6 sec      | 950 MB        | 77 KB      |
-| 2<sup>18</sup>  |  85 ms         | 15.5 sec     | 3.7 GB        | 89 KB      |
-| 2<sup>20</sup>  |  310 ms        | 67 sec       | 14 GB         | 100 KB     |
+| 2<sup>10</sup>  |  1 ms          | 60 ms        | 20 MB         | 46 KB      |
+| 2<sup>12</sup>  |  2 ms          | 180 ms       | 52 MB         | 56 KB      |
+| 2<sup>14</sup>  |  8 ms          | 680 ms       | 240 MB        | 65 KB      |
+| 2<sup>16</sup>  |  28 ms         | 2.7 sec      | 950 MB        | 75 KB      |
+| 2<sup>18</sup>  |  81 ms         | 11.4 sec     | 3.7 GB        | 87 KB      |
+| 2<sup>20</sup>  |  310 ms        | 47.5 sec     | 14 GB         | 100 KB     |
 
 As can be seen from the above, proving time roughly doubles with every doubling in the number of cycles, but proof size grows much slower.
 
 We can also generate proofs at a higher security level. The cost of doing so is roughly doubling of proving time and roughly 40% increase in proof size. In the benchmarks below, the same Fibonacci calculator program was executed on Apple M1 Pro CPU at 128-bit target security level:
 
 | VM cycles       | Execution time | Proving time | RAM consumed  | Proof size |
 | :-------------: | :------------: | :----------: | :-----------: | :--------: |
-| 2<sup>10</sup>  | 1 ms           | 300 ms       | 30 MB         | 61 KB      |
-| 2<sup>12</sup>  | 2 ms           | 590 ms       | 106 MB        | 78 KB      |
-| 2<sup>14</sup>  | 8 ms           | 1.7 sec      | 500 MB        | 91 KB      |
-| 2<sup>16</sup>  | 28 ms          | 6.7 sec      | 2.0 GB        | 106 KB     |
-| 2<sup>18</sup>  | 85 ms          | 27.5 sec     | 8.0 GB        | 122 KB     |
-| 2<sup>20</sup>  | 310 ms         | 126 sec      | 24.0 GB       | 138 KB     |
+| 2<sup>10</sup>  | 1 ms           | 120 ms       | 30 MB         | 61 KB      |
+| 2<sup>12</sup>  | 2 ms           | 460 ms       | 106 MB        | 77 KB      |
+| 2<sup>14</sup>  | 8 ms           | 1.4 sec      | 500 MB        | 90 KB      |
+| 2<sup>16</sup>  | 27 ms          | 4.9 sec      | 2.0 GB        | 103 KB     |
+| 2<sup>18</sup>  | 81 ms          | 20.1 sec     | 8.0 GB        | 121 KB     |
+| 2<sup>20</sup>  | 310 ms         | 90.3 sec     | 20.0 GB       | 138 KB     |
 
 ### Multi-core prover performance
-STARK proof generation is massively parallelizable. Thus, by taking advantage of multiple CPU cores we can dramatically reduce proof generation time. For example, when executed on an 8-core CPU (Apple M1 Pro), the current version of Miden VM operates at around 100 KHz. And when executed on a 64-core CPU (Amazon Graviton 3), the VM operates at around 250 KHz.
+STARK proof generation is massively parallelizable. Thus, by taking advantage of multiple CPU cores we can dramatically reduce proof generation time. For example, when executed on an 8-core CPU (Apple M1 Pro), the current version of Miden VM operates at around 140 KHz. And when executed on a 64-core CPU (Amazon Graviton 3), the VM operates at around 250 KHz.
 
 In the benchmarks below, the VM executes the same Fibonacci calculator program for 2<sup>20</sup> cycles at 96-bit target security level:
 
-| Machine                        | Execution time | Proving time | Execution % |
-| ------------------------------ | :------------: | :----------: | :---------: |
-| Apple M1 Pro (8 threads)       | 310 ms         | 9.8 sec      | 3.1%        |
-| Apple M2 Max (16 threads)      | 290 ms         | 7.7 sec      | 3.6%        |
-| AMD Ryzen 9 5950X (16 threads) | 270 ms         | 10.7 sec     | 2.6%        |
-| Amazon Graviton 3 (64 threads) | 330 ms         | 3.7 sec      | 9.0%        |
+| Machine                        | Execution time | Proving time | Execution % | Implied Frequency |
+| ------------------------------ | :------------: | :----------: | :---------: | :---------------: |
+| Apple M1 Pro (16 threads)      | 310 ms         | 7.0 sec      | 4.2%        | 140 KHz           |
+| Apple M2 Max (16 threads)      | 280 ms         | 5.8 sec      | 4.5%        | 170 KHz           |
+| AMD Ryzen 9 5950X (16 threads) | 270 ms         | 10.0 sec     | 2.6%        | 100 KHz           |
+| Amazon Graviton 3 (64 threads) | 330 ms         | 3.6 sec      | 8.5%        | 265 KHz           |
+
+### Recursive proofs
+Proofs in the above benchmarks are generated using BLAKE3 hash function. While this hash function is very fast, it is not very efficient to execute in Miden VM. Thus, proofs generated using BLAKE3 are not well-suited for recursive proof verification. To support efficient recursive proofs, we need to use an arithmetization-friendly hash function. Miden VM natively supports Rescue Prime Optimized (RPO), which is one such hash function. One of the downsides of arithmetization-friendly hash functions is that they are considerably slower than regular hash functions.
+
+In the benchmarks below we execute the same Fibonacci calculator program for 2<sup>20</sup> cycles at 96-bit target security level using RPO hash function instead of BLAKE3:
+
+| Machine                        | Execution time | Proving time | Proving time (HW) |
+| ------------------------------ | :------------: | :----------: | :---------------: |
+| Apple M1 Pro (16 threads)      | 310 ms         | 94.3 sec     | 42.0 sec          |
+| Apple M2 Max (16 threads)      | 280 ms         | 75.1 sec     | 20.9 sec          |
+| AMD Ryzen 9 5950X (16 threads) | 270 ms         | 59.3 sec     |                   |
+| Amazon Graviton 3 (64 threads) | 330 ms         | 21.7 sec     | 14.9 sec          |
+
+In the above, proof generation on some platforms can be hardware-accelerated. Specifically:
+
+* On Apple M1/M2 platforms the built-in GPU is used for a part of proof generation process.
+* On the Graviton platform, SVE vector extension is used to accelerate RPO computations.
 
 ## References
 Proofs of execution generated by Miden VM are based on STARKs. A STARK is a novel proof-of-computation scheme that allows you to create an efficiently verifiable proof that a computation was executed correctly. The scheme was developed by Eli Ben-Sasson, Michael Riabzev et al. at Technion - Israel Institute of Technology. STARKs do not require an initial trusted setup, and rely on very few cryptographic assumptions.

air/Cargo.toml

@@ -1,6 +1,6 @@
 [package]
 name = "miden-air"
-version = "0.6.0"
+version = "0.7.0"
 description = "Algebraic intermediate representation of Miden VM processor"
 authors = ["miden contributors"]
 readme = "README.md"
@@ -9,7 +9,7 @@ repository = "https://github.com/0xPolygonMiden/miden-vm"
 categories = ["cryptography", "no-std"]
 keywords = ["air", "arithmetization", "crypto", "miden"]
 edition = "2021"
-rust-version = "1.67"
+rust-version = "1.73"
 
 [lib]
 bench = false
@@ -28,10 +28,10 @@ default = ["std"]
 std = ["vm-core/std", "winter-air/std"]
 
 [dependencies]
-vm-core = { package = "miden-core", path = "../core", version = "0.6", default-features = false }
+vm-core = { package = "miden-core", path = "../core", version = "0.7", default-features = false }
 winter-air = { package = "winter-air", version = "0.6", default-features = false }
 
 [dev-dependencies]
 criterion = "0.5"
-proptest = "1.1"
+proptest = "1.3"
 rand-utils = { package = "winter-rand-utils", version = "0.6" }

air/benches/enforce_stack_constraint.rs

@@ -8,7 +8,7 @@ use miden_air::{
     Felt, FieldElement,
 };
 use std::time::Duration;
-use vm_core::Operation;
+use vm_core::{Operation, ZERO};
 
 fn enforce_stack_constraint(c: &mut Criterion) {
     let mut group = c.benchmark_group("enforce_stack_constraint");
@@ -27,7 +27,7 @@ fn enforce_stack_constraint(c: &mut Criterion) {
         frame.current_mut()[STACK_TRACE_OFFSET] = Felt::new(89u64);
         frame.next_mut()[STACK_TRACE_OFFSET] = Felt::new(89u64).inv();
 
-        let mut result = [Felt::ZERO; NUM_CONSTRAINTS];
+        let mut result = [ZERO; NUM_CONSTRAINTS];
 
         let frame = generate_evaluation_frame(36);
         bench.iter(|| {

air/src/constraints/chiplets/bitwise/mod.rs

@@ -6,6 +6,7 @@ use crate::{
         BITWISE_OUTPUT_COL_IDX, BITWISE_PREV_OUTPUT_COL_IDX, BITWISE_SELECTOR_COL_IDX,
     },
     utils::{are_equal, binary_not, is_binary, is_zero, EvaluationResult},
+    ONE, ZERO,
 };
 use winter_air::TransitionConstraintDegree;
 
@@ -413,27 +414,9 @@ pub fn agg_bits<E: FieldElement>(row: &[E], start_idx: usize) -> E {
 
 // CYCLE MASKS
 // ================================================================================================
-pub const BITWISE_K0_MASK: [Felt; OP_CYCLE_LEN] = [
-    Felt::ONE,
-    Felt::ZERO,
-    Felt::ZERO,
-    Felt::ZERO,
-    Felt::ZERO,
-    Felt::ZERO,
-    Felt::ZERO,
-    Felt::ZERO,
-];
-
-pub const BITWISE_K1_MASK: [Felt; OP_CYCLE_LEN] = [
-    Felt::ONE,
-    Felt::ONE,
-    Felt::ONE,
-    Felt::ONE,
-    Felt::ONE,
-    Felt::ONE,
-    Felt::ONE,
-    Felt::ZERO,
-];
+pub const BITWISE_K0_MASK: [Felt; OP_CYCLE_LEN] = [ONE, ZERO, ZERO, ZERO, ZERO, ZERO, ZERO, ZERO];
+
+pub const BITWISE_K1_MASK: [Felt; OP_CYCLE_LEN] = [ONE, ONE, ONE, ONE, ONE, ONE, ONE, ZERO];
 
 // TEST HELPERS
 // ================================================================================================
@@ -442,8 +425,8 @@ pub const BITWISE_K1_MASK: [Felt; OP_CYCLE_LEN] = [
 #[cfg(test)]
 fn get_periodic_values(cycle_row: usize) -> [Felt; 2] {
     match cycle_row {
-        0 => [Felt::ONE, Felt::ONE],
-        8 => [Felt::ZERO, Felt::ZERO],
-        _ => [Felt::ZERO, Felt::ONE],
+        0 => [ONE, ONE],
+        8 => [ZERO, ZERO],
+        _ => [ZERO, ONE],
     }
 }