From fd87cc86ba90afc659afe5aeb74fb3327f36f5d0 Mon Sep 17 00:00:00 2001 From: Bobbin Threadbare Date: Tue, 10 Oct 2023 23:58:55 -0700 Subject: [PATCH] docs: update readmes and docs for v0.7 release --- README.md | 59 +++++++++++++------- docs/src/SUMMARY.md | 1 + docs/src/intro/main.md | 2 +- docs/src/intro/overview.md | 2 +- docs/src/intro/performance.md | 55 +++++++++++------- docs/src/intro/usage.md | 15 ++++- docs/src/tools/main.md | 2 +- docs/src/user_docs/assembly/io_operations.md | 1 + docs/src/user_docs/stdlib/crypto/dsa.md | 15 +++++ miden/Cargo.toml | 1 + miden/README.md | 58 +++++++++++-------- processor/README.md | 12 ++-- prover/Cargo.toml | 2 +- prover/README.md | 8 ++- 14 files changed, 156 insertions(+), 77 deletions(-) create mode 100644 docs/src/user_docs/stdlib/crypto/dsa.md diff --git a/README.md b/README.md index 5b726873a3..a5df44b8ba 100644 --- a/README.md +++ b/README.md @@ -31,7 +31,7 @@ Miden VM is a fully-featured virtual machine. Despite being optimized for zero-k * **Cryptographic operations.** Miden assembly provides built-in instructions for computing hashes and verifying Merkle paths. These instructions use the Rescue Prime Optimized hash function (which is the native hash function of the VM). * **External libraries.** Miden VM supports compiling programs against pre-defined libraries. The VM ships with one such library: Miden `stdlib` which adds support for such things as 64-bit unsigned integers. Developers can build other similar libraries to extend the VM's functionality in ways which fit their use cases. * **Nondeterminism**. Unlike traditional virtual machines, Miden VM supports nondeterministic programming. This means a prover may do additional work outside of the VM and then provide execution *hints* to the VM. These hints can be used to dramatically speed up certain types of computations, as well as to supply secret inputs to the VM. -* **Custom advice providers.** Miden VM can be instantiated with user-defined advice providers. These advice providers are used to supply external data to the VM during execution/proof generation (via nondeterministic inputs) and can connect the VM to arbitrary data sources (e.g., a database or RPC calls). +* **Customizable hosts.** Miden VM can be instantiated with user-defined hosts. These hosts are used to supply external data to the VM during execution/proof generation (via nondeterministic inputs) and can connect the VM to arbitrary data sources (e.g., a database or RPC calls). #### Planned features In the coming months we plan to finalize the design of the VM and implement support for the following features: @@ -78,16 +78,16 @@ A few general notes on performance: * Both proof generation and proof verification times are greatly influenced by the hash function used in the STARK protocol. In the benchmarks below, we use BLAKE3, which is a really fast hash function. ### Single-core prover performance -When executed on a single CPU core, the current version of Miden VM operates at around 10 - 15 KHz. In the benchmarks below, the VM executes a [Fibonacci calculator](miden/README.md#fibonacci-calculator) program on Apple M1 Pro CPU in a single thread. The generated proofs have a target security level of 96 bits. +When executed on a single CPU core, the current version of Miden VM operates at around 20 - 25 KHz. In the benchmarks below, the VM executes a [Fibonacci calculator](miden/README.md#fibonacci-calculator) program on Apple M1 Pro CPU in a single thread. The generated proofs have a target security level of 96 bits. | VM cycles | Execution time | Proving time | RAM consumed | Proof size | | :-------------: | :------------: | :----------: | :-----------: | :--------: | -| 210 | 1 ms | 80 ms | 20 MB | 47 KB | -| 212 | 2 ms | 260 ms | 52 MB | 57 KB | -| 214 | 8 ms | 0.9 sec | 240 MB | 66 KB | -| 216 | 28 ms | 4.6 sec | 950 MB | 77 KB | -| 218 | 85 ms | 15.5 sec | 3.7 GB | 89 KB | -| 220 | 310 ms | 67 sec | 14 GB | 100 KB | +| 210 | 1 ms | 60 ms | 20 MB | 46 KB | +| 212 | 2 ms | 180 ms | 52 MB | 56 KB | +| 214 | 8 ms | 680 ms | 240 MB | 65 KB | +| 216 | 28 ms | 2.7 sec | 950 MB | 75 KB | +| 218 | 81 ms | 11.4 sec | 3.7 GB | 87 KB | +| 220 | 310 ms | 47.5 sec | 14 GB | 100 KB | As can be seen from the above, proving time roughly doubles with every doubling in the number of cycles, but proof size grows much slower. @@ -95,24 +95,41 @@ We can also generate proofs at a higher security level. The cost of doing so is | VM cycles | Execution time | Proving time | RAM consumed | Proof size | | :-------------: | :------------: | :----------: | :-----------: | :--------: | -| 210 | 1 ms | 300 ms | 30 MB | 61 KB | -| 212 | 2 ms | 590 ms | 106 MB | 78 KB | -| 214 | 8 ms | 1.7 sec | 500 MB | 91 KB | -| 216 | 28 ms | 6.7 sec | 2.0 GB | 106 KB | -| 218 | 85 ms | 27.5 sec | 8.0 GB | 122 KB | -| 220 | 310 ms | 126 sec | 24.0 GB | 138 KB | +| 210 | 1 ms | 120 ms | 30 MB | 61 KB | +| 212 | 2 ms | 460 ms | 106 MB | 77 KB | +| 214 | 8 ms | 1.4 sec | 500 MB | 90 KB | +| 216 | 27 ms | 4.9 sec | 2.0 GB | 103 KB | +| 218 | 81 ms | 20.1 sec | 8.0 GB | 121 KB | +| 220 | 310 ms | 90.3 sec | 20.0 GB | 138 KB | ### Multi-core prover performance -STARK proof generation is massively parallelizable. Thus, by taking advantage of multiple CPU cores we can dramatically reduce proof generation time. For example, when executed on an 8-core CPU (Apple M1 Pro), the current version of Miden VM operates at around 100 KHz. And when executed on a 64-core CPU (Amazon Graviton 3), the VM operates at around 250 KHz. +STARK proof generation is massively parallelizable. Thus, by taking advantage of multiple CPU cores we can dramatically reduce proof generation time. For example, when executed on an 8-core CPU (Apple M1 Pro), the current version of Miden VM operates at around 140 KHz. And when executed on a 64-core CPU (Amazon Graviton 3), the VM operates at around 250 KHz. In the benchmarks below, the VM executes the same Fibonacci calculator program for 220 cycles at 96-bit target security level: -| Machine | Execution time | Proving time | Execution % | -| ------------------------------ | :------------: | :----------: | :---------: | -| Apple M1 Pro (8 threads) | 310 ms | 9.8 sec | 3.1% | -| Apple M2 Max (16 threads) | 290 ms | 7.7 sec | 3.6% | -| AMD Ryzen 9 5950X (16 threads) | 270 ms | 10.7 sec | 2.6% | -| Amazon Graviton 3 (64 threads) | 330 ms | 3.7 sec | 9.0% | +| Machine | Execution time | Proving time | Execution % | Implied Frequency | +| ------------------------------ | :------------: | :----------: | :---------: | :---------------: | +| Apple M1 Pro (16 threads) | 310 ms | 7.0 sec | 4.2% | 140 KHz | +| Apple M2 Max (16 threads) | 280 ms | 5.8 sec | 4.5% | 170 KHz | +| AMD Ryzen 9 5950X (16 threads) | 270 ms | 10.0 sec | 2.6% | 100 KHz | +| Amazon Graviton 3 (64 threads) | 330 ms | 3.6 sec | 8.5% | 265 KHz | + +### Recursive proofs +Proofs in the above benchmarks are generated using BLAKE3 hash function. While this hash function is very fast, it is not very efficient to execute in Miden VM. Thus, proofs generated using BLAKE3 are not well-suited for recursive proof verification. To support efficient recursive proofs, we need to use an arithmetization-friendly hash function. Miden VM natively supports Rescue Prime Optimized (RPO), which is one such hash function. One of the downsides of arithmetization-friendly hash functions is that they are considerably slower than regular hash functions. + +In the benchmarks below we execute the same Fibonacci calculator program for 220 cycles at 96-bit target security level using RPO hash function instead of BLAKE3: + +| Machine | Execution time | Proving time | Proving time (HW) | +| ------------------------------ | :------------: | :----------: | :---------------: | +| Apple M1 Pro (16 threads) | 310 ms | 94.3 sec | 42.0 sec | +| Apple M2 Max (16 threads) | 280 ms | 75.1 sec | 20.9 sec | +| AMD Ryzen 9 5950X (16 threads) | 270 ms | 59.3 sec | | +| Amazon Graviton 3 (64 threads) | 330 ms | 21.7 sec | 14.9 sec | + +In the above, proof generation on some platforms can be hardware-accelerated. Specifically: + +* On Apple M1/M2 platforms the built-in GPU is used for a part of proof generation process. +* On the Graviton platform, SVE vector extension is used to accelerate RPO computations. ## References Proofs of execution generated by Miden VM are based on STARKs. A STARK is a novel proof-of-computation scheme that allows you to create an efficiently verifiable proof that a computation was executed correctly. The scheme was developed by Eli Ben-Sasson, Michael Riabzev et al. at Technion - Israel Institute of Technology. STARKs do not require an initial trusted setup, and rely on very few cryptographic assumptions. diff --git a/docs/src/SUMMARY.md b/docs/src/SUMMARY.md index 7b162b78a1..b344da23ee 100644 --- a/docs/src/SUMMARY.md +++ b/docs/src/SUMMARY.md @@ -20,6 +20,7 @@ - [Debugging](./user_docs/assembly/debugging.md) - [Miden Standard Library](./user_docs/stdlib/main.md) - [std::collections](./user_docs/stdlib/collections.md) + - [std::crypto::dsa](./user_docs/stdlib/crypto/dsa.md) - [std::crypto::fri](./user_docs/stdlib/crypto/fri.md) - [std::crypto::hashes](./user_docs/stdlib/crypto/hashes.md) - [std::math::u64](./user_docs/stdlib/math/u64.md) diff --git a/docs/src/intro/main.md b/docs/src/intro/main.md index b92e2ac454..05a3d44274 100644 --- a/docs/src/intro/main.md +++ b/docs/src/intro/main.md @@ -17,7 +17,7 @@ Miden VM is a fully-featured virtual machine. Despite being optimized for zero-k * **Cryptographic operations.** Miden assembly provides built-in instructions for computing hashes and verifying Merkle paths. These instructions use Rescue Prime Optimized hash function (which is the native hash function of the VM). * **External libraries.** Miden VM supports compiling programs against pre-defined libraries. The VM ships with one such library: Miden `stdlib` which adds support for such things as 64-bit unsigned integers. Developers can build other similar libraries to extend the VM's functionality in ways which fit their use cases. * **Nondeterminism**. Unlike traditional virtual machines, Miden VM supports nondeterministic programming. This means a prover may do additional work outside of the VM and then provide execution *hints* to the VM. These hints can be used to dramatically speed up certain types of computations, as well as to supply secret inputs to the VM. -* **Custom advice providers.** Miden VM can be instantiated with user-defined advice providers. These advice providers are used to supply external data to the VM during execution/proof generation (via nondeterministic inputs) and can connect the VM to arbitrary data sources (e.g., a database or RPC calls). +* **Customizable hosts.** Miden VM can be instantiated with user-defined hosts. These hosts are used to supply external data to the VM during execution/proof generation (via nondeterministic inputs) and can connect the VM to arbitrary data sources (e.g., a database or RPC calls). ### Planned features In the coming months we plan to finalize the design of the VM and implement support for the following features: diff --git a/docs/src/intro/overview.md b/docs/src/intro/overview.md index 8772e5ec9c..b7a5216968 100644 --- a/docs/src/intro/overview.md +++ b/docs/src/intro/overview.md @@ -8,7 +8,7 @@ Miden VM consists of three high-level components as illustrated below. These components are: * **Stack** which is a push-down stack where each item is a field element. Most assembly instructions operate with values located on the stack. The stack can grow up to $2^{32}$ items deep, however, only the top 16 items are directly accessible. * **Memory** which is a linear random-access read-write memory. The memory is word-addressable, meaning, four elements are located at each address, and we can read and write elements to/from memory in batches of four. Memory addresses can be in the range $[0, 2^{32})$. -* **Advice provider** which is a way for the prover to provide nondeterministic inputs to the VM. The advice provider is composed of a single *advice stack*, an *advice map*, and a *merkle store*. The *advice stack* yields elements to the VM stack; the *advice map* stores key-mapped element lists which can be pushed onto the advice stack; finally, the Merkle store contains structured Merkle tree data and serves Merkle paths to the VM. +* **Host** which is a way for the prover to provide nondeterministic inputs to the VM. The advice provider is composed of a single *advice stack*, an *advice map*, and a *merkle store*. The *advice stack* yields elements to the VM stack; the *advice map* stores key-mapped element lists which can be pushed onto the advice stack; finally, the Merkle store contains structured Merkle tree data and serves Merkle paths to the VM. In the future, additional components (e.g., storage, logs) may be added to the VM. diff --git a/docs/src/intro/performance.md b/docs/src/intro/performance.md index ab02a04fd7..9a481795a9 100644 --- a/docs/src/intro/performance.md +++ b/docs/src/intro/performance.md @@ -13,16 +13,16 @@ A few general notes on performance: * Both proof generation and proof verification times are greatly influenced by the hash function used in the STARK protocol. In the benchmarks below, we use BLAKE3, which is a really fast hash function. ## Single-core prover performance -When executed on a single CPU core, the current version of Miden VM operates at around 10 - 15 KHz. In the benchmarks below, the VM executes a Fibonacci calculator program on Apple M1 Pro CPU in a single thread. The generated proofs have a target security level of 96 bits. +When executed on a single CPU core, the current version of Miden VM operates at around 20 - 25 KHz. In the benchmarks below, the VM executes a Fibonacci calculator program on Apple M1 Pro CPU in a single thread. The generated proofs have a target security level of 96 bits. | VM cycles | Execution time | Proving time | RAM consumed | Proof size | | :-------------: | :------------: | :----------: | :-----------: | :--------: | -| 210 | 1 ms | 80 ms | 20 MB | 47 KB | -| 212 | 2 ms | 260 ms | 52 MB | 57 KB | -| 214 | 8 ms | 0.9 sec | 240 MB | 66 KB | -| 216 | 28 ms | 4.6 sec | 950 MB | 77 KB | -| 218 | 85 ms | 15.5 sec | 3.7 GB | 89 KB | -| 220 | 310 ms | 67 sec | 14 GB | 100 KB | +| 210 | 1 ms | 60 ms | 20 MB | 46 KB | +| 212 | 2 ms | 180 ms | 52 MB | 56 KB | +| 214 | 8 ms | 680 ms | 240 MB | 65 KB | +| 216 | 28 ms | 2.7 sec | 950 MB | 75 KB | +| 218 | 81 ms | 11.4 sec | 3.7 GB | 87 KB | +| 220 | 310 ms | 47.5 sec | 14 GB | 100 KB | As can be seen from the above, proving time roughly doubles with every doubling in the number of cycles, but proof size grows much slower. @@ -30,21 +30,38 @@ We can also generate proofs at a higher security level. The cost of doing so is | VM cycles | Execution time | Proving time | RAM consumed | Proof size | | :-------------: | :------------: | :----------: | :-----------: | :--------: | -| 210 | 1 ms | 300 ms | 30 MB | 61 KB | -| 212 | 2 ms | 590 ms | 106 MB | 78 KB | -| 214 | 8 ms | 1.7 sec | 500 MB | 91 KB | -| 216 | 28 ms | 6.7 sec | 2.0 GB | 106 KB | -| 218 | 85 ms | 27.5 sec | 8.0 GB | 122 KB | -| 220 | 310 ms | 126 sec | 24.0 GB | 138 KB | +| 210 | 1 ms | 120 ms | 30 MB | 61 KB | +| 212 | 2 ms | 460 ms | 106 MB | 77 KB | +| 214 | 8 ms | 1.4 sec | 500 MB | 90 KB | +| 216 | 27 ms | 4.9 sec | 2.0 GB | 103 KB | +| 218 | 81 ms | 20.1 sec | 8.0 GB | 121 KB | +| 220 | 310 ms | 90.3 sec | 20.0 GB | 138 KB | ## Multi-core prover performance STARK proof generation is massively parallelizable. Thus, by taking advantage of multiple CPU cores we can dramatically reduce proof generation time. For example, when executed on an 8-core CPU (Apple M1 Pro), the current version of Miden VM operates at around 100 KHz. And when executed on a 64-core CPU (Amazon Graviton 3), the VM operates at around 250 KHz. In the benchmarks below, the VM executes the same Fibonacci calculator program for 220 cycles at 96-bit target security level: -| Machine | Execution time | Proving time | Execution % | -| ------------------------------ | :------------: | :----------: | :---------: | -| Apple M1 Pro (8 threads) | 310 ms | 9.8 sec | 3.1% | -| Apple M2 Max (16 threads) | 290 ms | 7.7 sec | 3.6% | -| AMD Ryzen 9 5950X (16 threads) | 270 ms | 10.7 sec | 2.6% | -| Amazon Graviton 3 (64 threads) | 330 ms | 3.7 sec | 9.0% | +| Machine | Execution time | Proving time | Execution % | Implied Frequency | +| ------------------------------ | :------------: | :----------: | :---------: | :---------------: | +| Apple M1 Pro (16 threads) | 310 ms | 7.0 sec | 4.2% | 140 KHz | +| Apple M2 Max (16 threads) | 280 ms | 5.8 sec | 4.5% | 170 KHz | +| AMD Ryzen 9 5950X (16 threads) | 270 ms | 10.0 sec | 2.6% | 100 KHz | +| Amazon Graviton 3 (64 threads) | 330 ms | 3.6 sec | 8.5% | 265 KHz | + +### Recursive proofs +Proofs in the above benchmarks are generated using BLAKE3 hash function. While this hash function is very fast, it is not very efficient to execute in Miden VM. Thus, proofs generated using BLAKE3 are not well-suited for recursive proof verification. To support efficient recursive proofs, we need to use an arithmetization-friendly hash function. Miden VM natively supports Rescue Prime Optimized (RPO), which is one such hash function. One of the downsides of arithmetization-friendly hash functions is that they are considerably slower than regular hash functions. + +In the benchmarks below we execute the same Fibonacci calculator program for 220 cycles at 96-bit target security level using RPO hash function instead of BLAKE3: + +| Machine | Execution time | Proving time | Proving time (HW) | +| ------------------------------ | :------------: | :----------: | :---------------: | +| Apple M1 Pro (16 threads) | 310 ms | 94.3 sec | 42.0 sec | +| Apple M2 Max (16 threads) | 280 ms | 75.1 sec | 20.9 sec | +| AMD Ryzen 9 5950X (16 threads) | 270 ms | 59.3 sec | | +| Amazon Graviton 3 (64 threads) | 330 ms | 21.7 sec | 14.9 sec | + +In the above, proof generation on some platforms can be hardware-accelerated. Specifically: + +* On Apple M1/M2 platforms the built-in GPU is used for a part of proof generation process. +* On the Graviton platform, SVE vector extension is used to accelerate RPO computations. diff --git a/docs/src/intro/usage.md b/docs/src/intro/usage.md index 29185daa54..cfba00ade8 100644 --- a/docs/src/intro/usage.md +++ b/docs/src/intro/usage.md @@ -33,14 +33,23 @@ In this case, the `miden` executable will be placed in the `./target/release` di Internally, Miden VM uses [rayon](https://github.com/rayon-rs/rayon) for parallel computations. To control the number of threads used to generate a STARK proof, you can use `RAYON_NUM_THREADS` environment variable. ### GPU acceleration -Miden VM proof generation can be accelerated via GPUs. Currently, GPU acceleration is enabled only on Apple silicon hardware (via Metal). To compile Miden VM with Metal acceleration enabled, you can run the following command: +Miden VM proof generation can be accelerated via GPUs. Currently, GPU acceleration is enabled only on Apple silicon hardware (via [Metal](https://en.wikipedia.org/wiki/Metal_(API))). To compile Miden VM with Metal acceleration enabled, you can run the following command: ``` make exec-metal ``` -Similar to `make exec` command, this will place the resulting `miden` executable into the `./target/optimized directory`. +Similar to `make exec` command, this will place the resulting `miden` executable into the `./target/optimized` directory. -Currently, GPU acceleration is applicable only to recursive proofs which can be generated using `-r` flag. +Currently, GPU acceleration is applicable only to recursive proofs which can be generated using the `-r` flag. + +### SIMD acceleration +Miden VM execution and proof generation can be accelerated via vectorized instructions. Currently, SIMD acceleration can be enabled only on platforms supporting [SVE](https://en.wikipedia.org/wiki/AArch64#Scalable_Vector_Extension_(SVE)) instructions (e.g., Graviton 3). To compile Miden VM with SVE acceleration enabled, you can run the following command: +``` +make exec-graviton +``` +This will place the resulting `miden` executable into the `./target/optimized` directory. + +Similar to Metal acceleration, SVE acceleration is currently applicable only to recursive proofs which can be generated using the `-r` flag. ### Running Miden VM Once the executable has been compiled, you can run Miden VM like so: diff --git a/docs/src/tools/main.md b/docs/src/tools/main.md index 12859cffa0..4a98268e8e 100644 --- a/docs/src/tools/main.md +++ b/docs/src/tools/main.md @@ -2,7 +2,7 @@ The following tools are available for interacting with Miden VM: -* Via the [miden-vm](https://crates.io/crates/miden) crate (or within the Miden VM repo): +* Via the [miden-vm](https://crates.io/crates/miden-vm) crate (or within the Miden VM repo): * [CLI](../intro/usage.md#cli-interface) * [Debugger](./debugger.md) * [REPL](./repl.md) diff --git a/docs/src/user_docs/assembly/io_operations.md b/docs/src/user_docs/assembly/io_operations.md index 501c58f0da..f6f267e7b2 100644 --- a/docs/src/user_docs/assembly/io_operations.md +++ b/docs/src/user_docs/assembly/io_operations.md @@ -53,6 +53,7 @@ Advice injectors fall into two categories: (1) injectors which push new data ont | adv.push_mtnode | [d, i, R, ... ] | [d, i, R, ... ] | Pushes a node of a Merkle tree with root $R$ at depth $d$ and index $i$ from Merkle store onto the advice stack. | | adv.push_u64div | [b1, b0, a1, a0, ...] | [b1, b0, a1, a0, ...] | Pushes the result of `u64` division $a / b$ onto the advice stack. Both $a$ and $b$ are represented using 32-bit limbs. The result consists of both the quotient and the remainder. | | adv.push_ext2intt | [osize, isize, iptr, ... ] | [osize, isize, iptr, ... ] | Given evaluations of a polynomial over some specified domain, interpolates the evaluations into a polynomial in coefficient form and pushes the result into the advice stack. | +| adv.push_sig.*kind* | [K, M, ...] | [K, M, ...] | Pushes values onto the advice stack which are required for verification of a DSA with scheme specified by *kind* against the public key commitment $K$ and message $M$. | | adv.smt_get | [K, R, ... ] | [K, R, ... ] | Pushes values onto the advice stack which are required for successful retrieval of a value under the key $K$ from a Sparse Merkle Tree with root $R$. | | adv.smt_set | [V, K, R, ...] | [V, K, R, ...] | Pushes values onto the advice stack which are required for successful insertion of a key-value pair $(K, V)$ into a Sparse Merkle Tree with root $R$. | | adv.smt_peek | [K, R, ... ] | [K, R, ... ] | Pushes value onto the advice stack which is associated with key $K$ in a Sparse Merkle Tree with root $R$. | diff --git a/docs/src/user_docs/stdlib/crypto/dsa.md b/docs/src/user_docs/stdlib/crypto/dsa.md new file mode 100644 index 0000000000..f4c31b7185 --- /dev/null +++ b/docs/src/user_docs/stdlib/crypto/dsa.md @@ -0,0 +1,15 @@ +# Digital signatures +Namespace `std::crypto::dsa` contains a set of digital signature schemes supported by default in the Miden VM. Currently, these schemes are: + +* `RPO Falcon512`: a variant of the [Falcon](https://falcon-sign.info/) signature scheme. + +## RPO Falcon512 + +Module `std::crypto::dsa::rpo_falcon512` contains procedures for verifying `RPO Falcon512` signatures. These signatures differ from the standard Falcon signatures in that instead of using `SHAKE256` hash function in the *hash-to-point* algorithm we use `RPO256`. This makes the signature more efficient to verify in the Miden VM. + +The module exposes the following procedures: + +| Procedure | Description | +| ----------- | ------------- | +| verify | Verifies a signature against a public key and a message. The procedure gets as inputs the hash of the public key and the hash of the message via the operand stack. The signature is expected to be provided via the advice provider.

The signature is valid if and only if the procedure returns.

Inputs: `[PK, MSG, ...]`
Outputs: `[...]`

Where `PK` is the hash of the public key and `MSG` is the hash of the message. Both hashes are expected to be computed using `RPO` hash function.

The procedure relies on the `adv.push_sig` [decorator](../../assembly/io_operations.md#nondeterministic-inputs) to retrieve the signature from the host. The default host implementation assumes that the private-public key pair is loaded into the advice provider, and uses it to generate the signature. However, for production grade implementations, this functionality should be overridden to ensure more secure handling of private keys.| + diff --git a/miden/Cargo.toml b/miden/Cargo.toml index dc277ad5c3..fc94d382dc 100644 --- a/miden/Cargo.toml +++ b/miden/Cargo.toml @@ -40,6 +40,7 @@ path = "tests/integration/main.rs" concurrent = ["prover/concurrent", "std"] default = ["std"] executable = ["dep:env_logger", "dep:hex", "hex?/std", "std", "dep:serde", "serde?/std", "dep:serde_derive", "dep:serde_json", "serde_json?/std", "dep:clap", "dep:rustyline"] +metal = ["prover/metal", "std"] std = ["assembly/std", "log/std", "processor/std", "prover/std", "verifier/std"] sve = ["processor/sve", "prover/sve", "std"] diff --git a/miden/README.md b/miden/README.md index 639d6204ff..01793b87bb 100644 --- a/miden/README.md +++ b/miden/README.md @@ -1,11 +1,11 @@ # Miden VM -This crate aggregates all components of Miden VM in a single place. Specifically, it re-exports functionality from [processor](../processor/), [prover](../prover/), and [verifier](../verifier/) crates. Additionally, when compiled as an executable, this crate can be used via a [CLI interface](#cli-interface) to execute Miden VM programs and to verify correctness of their execution. +This crate aggregates all components of the Miden VM in a single place. Specifically, it re-exports functionality from [processor](../processor/), [prover](../prover/), and [verifier](../verifier/) crates. Additionally, when compiled as an executable, this crate can be used via a [CLI interface](#cli-interface) to execute Miden VM programs and to verify correctness of their execution. ## Basic concepts An in-depth description of Miden VM is available in the full Miden VM [documentation](https://0xpolygonmiden.github.io/miden-vm/). In this section we cover only the basics to make the included examples easier to understand. ### Writing programs -Our goal is to make Miden VM an easy compilation target for high-level blockchain-centric languages such as Solidity, Move, Sway, and others. We believe it is important to let people write programs in the languages of their choice. However, compilers to help with this have not been developed yet. Thus, for now, the primary way to write programs for Miden VM is to use [Miden assembly](../assembly). +Our goal is to make Miden VM an easy compilation target for high-level languages such as Rust, Move, Sway, and others. We believe it is important to let people write programs in the languages of their choice. However, compilers to help with this have not been developed yet. Thus, for now, the primary way to write programs for Miden VM is to use [Miden assembly](../assembly). Miden assembler compiles assembly source code in a [program MAST](https://0xpolygonmiden.github.io/miden-vm/design/programs.html), which is represented by a `Program` struct. It is possible to construct a `Program` struct manually, but we don't recommend this approach because it is tedious, error-prone, and requires an in-depth understanding of VM internals. All examples throughout these docs use assembly syntax. @@ -16,32 +16,33 @@ All Miden programs can be reduced to a single 32-byte value, called program hash Currently, there are 3 ways to get values onto the stack: 1. You can use `push` instruction to push values onto the stack. These values become a part of the program itself, and, therefore, cannot be changed between program executions. You can think of them as constants. -2. The stack can be initialized to some set of values at the beginning of the program. These inputs are public and must be shared with the verifier for them to verify a proof of the correct execution of a Miden program. The number of elements at the top of the stack which can receive an initial value is limited to 16. +2. The stack can be initialized to some set of values at the beginning of the program. These inputs are public and must be shared with the verifier for them to verify a proof of the correct execution of a Miden program. While it is possible to initialize the stack with a large number of values, we recommend limiting the number of initial values to a few doze at most as each initial value beyond 16 increases verifier complexity. 3. The program may request nondeterministic advice inputs from the prover. These inputs are secret inputs. This means that the prover does not need to share them with the verifier. There are three types of advice inputs: (1) a single advice stack which can contain any number of elements; (2) a key-mapped element lists which can be pushed onto the advice stack; (3) a Merkle store, which is used to provide nondeterministic inputs for instructions which work with Merkle trees. There are no restrictions on the number of advice inputs a program can request. -The stack is provided to Miden VM via `StackInputs` struct. These are public inputs of the execution, and should also be provided to the verifier. The secret inputs of the program are provided via `AdviceProvider` instances. There is one in-memory advice provider that can be commonly used for operations that won't require persistence: `MemAdviceProvider`. +The stack is provided to Miden VM via `StackInputs` struct. These are public inputs of the execution, and should also be provided to the verifier. The secret inputs for the program are provided via the `Host` interface. The default implementation of the host relies on in-memory advice provider (`MemAdviceProvider`) that can be commonly used for operations that won't require persistence. -Values remaining on the stack after a program is executed can be returned as stack outputs. You can specify exactly how many values (from the top of the stack) should be returned. Currently, the maximum number of outputs is limited to 16. +Values remaining on the stack after a program is executed can be returned as stack outputs. You can specify exactly how many values (from the top of the stack) should be returned. Similar to stack inputs, a large number of values can be returned via the stack, however, we recommend keeping this number as small as possible not to overburden the verifier. -Having only 16 elements to describe public inputs and outputs of a program may seem limiting, however, just 4 elements are sufficient to represent a root of a Merkle tree or a sequential hash of elements. Both of these can be expanded into an arbitrary number of values by supplying the actual values non-deterministically via the advice provider. +Having a small number elements to describe public inputs and outputs of a program may seem limiting, however, just 4 elements are sufficient to represent a root of a Merkle tree or a sequential hash of elements. Both of these can be expanded into an arbitrary number of values by supplying the actual values non-deterministically via the host interface. ## Usage Miden crate exposes several functions which can be used to execute programs, generate proofs of their correct execution, and verify the generated proofs. How to do this is explained below, but you can also take a look at working examples [here](examples) and find instructions for running them via CLI [here](#fibonacci-example). ### Executing programs -To execute a program on Miden VM, you can use either `execute()` or `execute_iter()` functions. Both of these functions take the same arguments: +To execute a program on Miden VM, you can use either `execute()` or `execute_iter()` functions. The `execute()` function takes the following arguments: * `program: &Program` - a reference to a Miden program to be executed. * `stack_inputs: StackInputs` - a set of public inputs with which to execute the program. -* `advice_provider: AdviceProvider` - an instance of an advice provider that yields secret, non-deterministic inputs to the prover. +* `host: Host` - an instance of a `Host` which can be used to supply non-deterministic inputs to the VM and receive messages from the VM. +* `options: ExecutionOptions` - a set of options for executing the specified program (e.g., max allowed number of cycles). -The `execute()` function returns a `Result` which will contain the execution trace of the program if the execution was successful, or an error, if the execution failed. You can inspect the trace to get the final state of the VM out of it, but generally, this trace is intended to be used internally by the prover during proof generation process. +The function returns a `Result` which will contain the execution trace of the program if the execution was successful, or an error, if the execution failed. Internally, the VM then passes this execution trace to the prover to generate a proof of a correct execution of the program. -The `execute_iter()` function returns a `VmStateIterator` which can be used to iterate over the cycles of the executed program for debug purposes. In fact, when we execute a program using this function, a lot of the debug information is retained and we can get a precise picture of the VM's state at any cycle. Moreover, if the execution results in an error, the `VmStateIterator` can still be used to inspect VM states right up to the cycle at which the error occurred. +The `execute_iter()` function takes similar arguments (but without the `options`) and returns a `VmStateIterator` . This iterator can be used to iterate over the cycles of the executed program for debug purposes. In fact, when we execute a program using this function, a lot of the debug information is retained and we can get a precise picture of the VM's state at any cycle. Moreover, if the execution results in an error, the `VmStateIterator` can still be used to inspect VM states right up to the cycle at which the error occurred. For example: ```rust -use miden::{Assembler, execute, execute_iter, MemAdviceProvider, DefaultHost, StackInputs}; +use miden::{Assembler, execute, execute_iter, DefaultHost, StackInputs}; use processor::ExecutionOptions; // instantiate the assembler @@ -53,7 +54,7 @@ let program = assembler.compile("begin push.3 push.5 add end").unwrap(); // use an empty list as initial stack let stack_inputs = StackInputs::default(); -// instantiate an empty advice provider +// instantiate a default host (with an empty advice provider) let mut host = DefaultHost::default(); // instantiate default execution options @@ -76,7 +77,7 @@ To execute a program on Miden VM and generate a proof that the program was execu * `program: &Program` - a reference to a Miden program to be executed. * `stack_inputs: StackInputs` - a set of public inputs with which to execute the program. -* `advice_provider: AdviceProvider` - an instance of an advice provider that yields secret, non-deterministic inputs to the prover. +* `host: Host` - an instance of a `Host` which can be used to supply non-deterministic inputs to the VM and receive messages from the VM. * `options: ProvingOptions` - config parameters for proof generation. The default options target 96-bit security level. If the program is executed successfully, the function returns a tuple with 2 elements: @@ -99,7 +100,7 @@ let program = assembler.compile("begin push.3 push.5 add end").unwrap(); let (outputs, proof) = prove( &program, StackInputs::default(), // we won't provide any inputs - DefaultHost::default(), // we won't provide advice inputs + DefaultHost::default(), // we'll be using a default host ProvingOptions::default(), // we'll be using default options ) .unwrap(); @@ -176,7 +177,7 @@ let source = format!( ); let program = Assembler::default().compile(&source).unwrap(); -// initialize an empty advice provider +// initialize a default host (with an empty advice provider) let host = DefaultHost::default(); // initialize the stack with values 0 and 1 @@ -207,19 +208,30 @@ First, make sure you have Rust [installed](https://www.rust-lang.org/tools/insta Then, to compile Miden VM into a binary, run the following command: ```shell -cargo build --release --features executable +cargo build --profile optimized --features executable ``` -This will place `miden` executable in the `./target/release` directory. +This will place `miden` executable in the `./target/optimized` directory. By default, the executable will be compiled in the single-threaded mode. If you would like to enable multi-threaded proof generation, you can compile Miden VM using the following command: ```shell -cargo build --release --features "executable concurrent" +cargo build --profile optimized --features concurrent,executable +``` +We also provide a number of `make` commands to simplify building Miden VM for various targets: +```shell +# build an executable for a generic target (concurrent) +make exec + +# build an executable for Apple silicon (concurrent+metal) +make exec-metal + +# built an executable for the Graviton 3 target (concurrent+sve) +make exec-graviton ``` ### Running Miden VM Once the executable has been compiled, you can run Miden VM like so: ```shell -./target/release/miden [subcommand] [parameters] +./target/optimized/miden [subcommand] [parameters] ``` Currently, Miden VM can be executed with the following subcommands: * `run` - this will execute a Miden assembly program and output the result, but will not generate a proof of execution. @@ -231,17 +243,17 @@ Currently, Miden VM can be executed with the following subcommands: All of the above subcommands require various parameters to be provided. To get more detailed help on what is needed for a given subcommand, you can run the following: ```shell -./target/release/miden [subcommand] --help +./target/optimized/miden [subcommand] --help ``` For example: ```shell -./target/release/miden prove --help +./target/optimized/miden prove --help ``` ### Fibonacci example In the `miden/examples/fib` directory, we provide a very simple Fibonacci calculator example. This example computes the 1000th term of the Fibonacci sequence. You can execute this example on Miden VM like so: ```shell -./target/release/miden run -a miden/examples/fib/fib.masm -n 1 +./target/optimized/miden run -a miden/examples/fib/fib.masm -n 1 ``` This will run the example code to completion and will output the top element remaining on the stack. @@ -251,6 +263,8 @@ Miden VM can be compiled with the following features: * `std` - enabled by default and relies on the Rust standard library. * `concurrent` - implies `std` and also enables multi-threaded proof generation. * `executable` - required for building Miden VM binary as described above. Implies `std`. +* `sve` - enables [SVE](https://en.wikipedia.org/wiki/AArch64#Scalable_Vector_Extension_(SVE))-based acceleration of the RPO hash function on supported platforms (e.g., Graviton 3). +* `metal` - enables [Metal](https://en.wikipedia.org/wiki/Metal_(API))-based acceleration of proof generation (for recursive proofs) on supported platforms (e.g., Apple silicon). * `no_std` does not rely on the Rust standard library and enables compilation to WebAssembly. To compile with `no_std`, disable default features via `--no-default-features` flag. diff --git a/processor/README.md b/processor/README.md index 5c34d42505..d176d158e1 100644 --- a/processor/README.md +++ b/processor/README.md @@ -2,15 +2,16 @@ This crate contains an implementation of Miden VM processor. The purpose of the processor is to execute a program and to generate a program execution trace. This trace is then used by Miden VM to generate a proof of correct execution of the program. ## Usage -The processor exposes two functions which can be used to execute programs: `execute()` and `execute_iter()`. Both of these functions take the same arguments: +The processor exposes two functions which can be used to execute programs: `execute()` and `execute_iter()`. The `execute()` function takes the following arguments: * `program: &Program` - a reference to a Miden program to be executed. * `stack_inputs: StackInputs` - a set of public inputs with which to execute the program. -* `advice_provider: AdviceProvider` - an instance of an advice provider that yields secret, non-deterministic inputs to the prover. +* `host: Host` - an instance of a `Host` which can be used to supply non-deterministic inputs to the VM and receive messages from the VM. +* `options: ExecutionOptions` - a set of options for executing the specified program (e.g., max allowed number of cycles). -The `execute()` function returns a `Result` which will contain the execution trace of the program if the execution was successful, or an error, if the execution failed. Internally, the VM then passes this execution trace to the prover to generate a proof of a correct execution of the program. +The function returns a `Result` which will contain the execution trace of the program if the execution was successful, or an error, if the execution failed. Internally, the VM then passes this execution trace to the prover to generate a proof of a correct execution of the program. -The `execute_iter()` function returns a `VmStateIterator` which can be used to iterate over the cycles of the executed program for debug purposes. In fact, when we execute a program using this function, a lot of the debug information is retained and we can get a precise picture of the VM's state at any cycle. Moreover, if the execution results in an error, the `VmStateIterator` can still be used to inspect VM states right up to the cycle at which the error occurred. +The `execute_iter()` function takes similar arguments (but without the `options`) and returns a `VmStateIterator` . This iterator can be used to iterate over the cycles of the executed program for debug purposes. In fact, when we execute a program using this function, a lot of the debug information is retained and we can get a precise picture of the VM's state at any cycle. Moreover, if the execution results in an error, the `VmStateIterator` can still be used to inspect VM states right up to the cycle at which the error occurred. For example: ```Rust @@ -26,7 +27,7 @@ let program = assembler.compile("begin push.3 push.5 add end").unwrap(); // use an empty list as initial stack let stack_inputs = StackInputs::default(); -// instantiate an empty advice provider +// instantiate a default host (with no advice inputs) let mut host = DefaultHost::default(); // instantiate default execution options @@ -61,6 +62,7 @@ A much more in-depth description of Miden VM design is available [here](https:// Miden processor can be compiled with the following features: * `std` - enabled by default and relies on the Rust standard library. +* `sve` - enables [SVE](https://en.wikipedia.org/wiki/AArch64#Scalable_Vector_Extension_(SVE))-based acceleration of the RPO hash function on supported platforms (e.g., Graviton 3). * `no_std` does not rely on the Rust standard library and enables compilation to WebAssembly. To compile with `no_std`, disable default features via `--no-default-features` flag. diff --git a/prover/Cargo.toml b/prover/Cargo.toml index 84fa93f5fa..32ddd66462 100644 --- a/prover/Cargo.toml +++ b/prover/Cargo.toml @@ -25,6 +25,6 @@ processor = { package = "miden-processor", path = "../processor", version = "0.7 winter-prover = { package = "winter-prover", version = "0.6", default-features = false } [target.'cfg(all(target_arch = "aarch64", target_os = "macos"))'.dependencies] -ministark-gpu = { version = "0.1", features = [ "winterfell" ], optional = true } elsa = { version = "1.9", optional = true } +ministark-gpu = { version = "0.1", features = [ "winterfell" ], optional = true } pollster = { version = "0.3", optional = true } diff --git a/prover/README.md b/prover/README.md index 5444ad050e..f707bb5f63 100644 --- a/prover/README.md +++ b/prover/README.md @@ -6,7 +6,7 @@ This crate exposes a `prove()` function which can be used to execute Miden VM pr * `program: &Program` - a reference to a Miden program to be executed. * `stack_inputs: StackInputs` - a set of public inputs with which to execute the program. -* `advice_provider: AdviceProvider` - an instance of an advice provider that yields secret, non-deterministic inputs to the prover. +* `host: Host` - an instance of a `Host` which can be used to supply non-deterministic inputs to the VM and receive messages from the VM. * `options: &ProvingOptions` - config parameters for proof generation. The default options target 96-bit security level. If the program is executed successfully, the function returns a tuple with 2 elements: @@ -30,8 +30,8 @@ let program = assembler.compile("begin push.3 push.5 add end").unwrap(); let (outputs, proof) = prove( &program, StackInputs::default(), // we won't provide any stack inputs - DefaultHost::default(), // we won't provide any advice values - &ProvingOptions::default(), // we'll be using default options + DefaultHost::default(), // we'll be using a default host + &ProvingOptions::default(), // we'll be using default options ) .unwrap(); @@ -44,6 +44,8 @@ Miden prover can be compiled with the following features: * `std` - enabled by default and relies on the Rust standard library. * `concurrent` - implies `std` and also enables multi-threaded proof generation. +* `sve` - enables [SVE](https://en.wikipedia.org/wiki/AArch64#Scalable_Vector_Extension_(SVE))-based acceleration of the RPO hash function on supported platforms (e.g., Graviton 3). +* `metal` - enables [Metal](https://en.wikipedia.org/wiki/Metal_(API))-based acceleration of proof generation (for recursive proofs) on supported platforms (e.g., Apple silicon). * `no_std` does not rely on the Rust standard library and enables compilation to WebAssembly. To compile with `no_std`, disable default features via `--no-default-features` flag.