Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

docs: Implement docs feedback for Chapter 1 #558

Merged
merged 7 commits into from
Nov 14, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 1 addition & 2 deletions docs/src/architecture/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,8 +22,7 @@ We do provide an implementation of the storage provider, you can read more about

![](../images/architecture/pallets_overview.png)

Going over to the pallets, we've focused on the core functionality —
to that end, we've implemented the market, storage provider, proofs and randomness pallets.
We've been focusing on implementing the core functionality by developing the market, storage provider, proofs and randomness pallets.

The market pallet handles all things related to deal payments and slashing,
being informed by the storage provider when deals haven't been proven and applying slashing in those cases.
Expand Down
8 changes: 4 additions & 4 deletions docs/src/architecture/pallets/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -28,12 +28,12 @@ After publishing, the funds allocated for the deal will be moved from `free` to
<img src="../../images/market/publish_storage_deals.svg" alt="Publishing storage deals">

At this point, the remaining responsibility is shifted to the storage provider, which needs to activate the deal.
First, the storage provider needs to get_randomness from [Randomness Pallet](./randomness.md) in order to create a replica and
First, the storage provider needs to call `get_randomness` from the[Randomness Pallet](./randomness.md) in order to create a replica and
[pre-commit](./storage-provider.md#pre_commit_sectors) the deal's sectors.
The sealing and pre-comitting takes some time, after that the storage provider needs to fetch yet another randomness seed to create a proof.
After that, they [prove](./storage-provider.md#prove_commit_sectors) they stored the sectors by calling [`prove_commit_sectors`](./storage-provider.md#prove_commit_sectors) extrinsics.
The sealing and pre-committing takes some time, after that the storage provider needs to fetch yet another randomness seed to create a proof.
Subsequently, they [prove](./storage-provider.md#prove_commit_sectors) they stored the sectors by calling [`prove_commit_sectors`](./storage-provider.md#prove_commit_sectors) extrinsics.

Verification is done via [Proofs Pallet](./proofs.md) and reported to the Market pallet to terminate the deal and apply penalties to the storage provider
Verification is done via the [Proofs Pallet](./proofs.md) and reported to the Market pallet to terminate the deal and apply penalties to the storage provider
(remove and burn its collateral — i.e. `locked` funds) if they fail to activate the deal on time and return the funds to the client.

<img src="../../images/storage-provider/sector_activation.svg" alt="Deal activation">
Expand Down
20 changes: 10 additions & 10 deletions docs/src/architecture/pallets/market.md
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@ The reserved amount will be considered `free` until it is used in a deal when

#### <a id="add_balance.example" href="#add_balance.example" class="header">Example</a>

Using the `storagext-cli` to add 1000000000[^existential_deposit] [Plancks](../../glossary.md#planck) to Alice's account with the following command[^add_balance]:
Using the `storagext-cli` to add 1_000_000_000[^existential_deposit] [Plancks](../../glossary.md#planck) to Alice's account with the following command[^add_balance]:

```bash
storagext-cli --sr25519-key "//Alice" market add-balance 1000000000
Expand All @@ -42,14 +42,14 @@ storagext-cli --sr25519-key "//Alice" market add-balance 1000000000
This value is the minimum amount due to Polkadot's existential deposit.
More information available in: <https://support.polkadot.network/support/solutions/articles/65000168651-what-is-the-existential-deposit->.

[^add_balance]: Read more about the `add-balance` command in [_Storagext CLI/Subcommand `market`/`add-balance`_](../../storagext-cli/market.md#add-balance)
[^add_balance]: Read more about the `add-balance` command in [_Storagext CLI/Subcommand `market`/`add-balance`_](../../storagext-cli/market.md#add-balance).

### `withdraw_balance`

Withdraws funds from the Storage Market.

The funds will be withdrawn from the `free` balance, meaning that the `amount` must be
lesser than or equal to `free` and greater than 0 (\\({free} \ge {amount} \gt 0\\)).
less than or equal to `free` and greater than 0 (\\({free} \ge {amount} \gt 0\\)).

| Name | Description | Type |
| -------- | -------------------------- | ---------------- |
Expand All @@ -63,7 +63,7 @@ Using the `storagext-cli` to withdraw 10000 [Plancks](../../glossary.md#planck)
storagext-cli --sr25519-key "//Alice" market withdraw-balance 10000
```

[^withdraw_balance]: Read more about the `withdraw-balance` command in [_Storagext CLI/Subcommand `market`/`withdraw-balance`_](../../storagext-cli/market.md#withdraw-balance)
[^withdraw_balance]: Read more about the `withdraw-balance` command in [_Storagext CLI/Subcommand `market`/`withdraw-balance`_](../../storagext-cli/market.md#withdraw-balance).

### `publish_storage_deals`

Expand All @@ -73,7 +73,7 @@ This extrinsic _must_ be called by a storage provider.

| Name | Description | Type |
| ------------------ | ----------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `proposal` | Specific deal proposal, a JSON object | JSON object, specified in the [next section](#deal-proposal-components) |
| `proposal` | Specific deal proposal, a JSON object | JSON object, specified in the [deal proposal components section](#deal-proposal-components) |
| `client_signature` | Client signature of this specific deal proposal | [`MultiSignature`](https://paritytech.github.io/polkadot-sdk/master/sp_runtime/enum.MultiSignature.html), meaning a 64-byte array for Sr25519 and Ed25519 signatures and 65-byte array for ECDSA signatures |

The `client_signature`, as the name indicates, is generated by the client by signing the deal proposal with their private key —
Expand All @@ -94,8 +94,8 @@ _This step corresponds to the "sign & send proposal" step in the [deal overview]
| `label` | Arbitrary client chosen label | String, with a maximum length of 128 characters |
| `start_block` | Block number on which the deal should start | Positive integer |
| `end_block` | Block number on which the deal should end | Positive integer, `end_block > start_block` |
| `storage_price_per_block` | Price for the storage specified per block[^storage_price_per_block] | Positive integer, in [Plancks](../../glossary.md#planck) |
| `provider_collateral` | Collateral which is slashed if the deal fails | Positive integer, in [Plancks](../../glossary.md#planck) |
| `storage_price_per_block` | Price for the storage specified per block[^storage_price_per_block] | Positive integer, in [Plancks](../../glossary.md#planck) |
| `provider_collateral` | Collateral which is slashed if the deal fails | Positive integer, in [Plancks](../../glossary.md#planck) |
| `state` | Deal state. Can only be set to `Published` | String |

See the [original Filecoin specification](https://spec.filecoin.io/#section-systems.filecoin_markets.onchain_storage_market.storage_deal_flow) for details.
Expand Down Expand Up @@ -149,9 +149,9 @@ and is **not final**; it is a testing tool.

</div>

[^storage_price_per_block]: The formula to calculate the total price is as follows: \\\[total\\\_price = (end\\\_block - start\\\_block) \* storage\\\_price\\\_per\\\_block\\]
[^publish_storage_deals]: Read more about the `publish-storage-deals` command in [_Storagext CLI/Subcommand `market`/`publish-storage-deals`_](../../storagext-cli/market.md#publish-storage-deals)
[^warn]: Take into account that the CLI is currently for demo purposes, the authors are aware that the command isn't safe since it requires the private keys of both parties.
[^storage_price_per_block]: The formula to calculate the total price is as follows: \\\[total\\\_price = (end\\\_block - start\\\_block) \* storage\\\_price\\\_per\\\_block\\].
[^publish_storage_deals]: Read more about the `publish-storage-deals` command in [_Storagext CLI/Subcommand `market`/`publish-storage-deals`_](../../storagext-cli/market.md#publish-storage-deals).

### `settle_deal_payments`

Expand Down Expand Up @@ -199,7 +199,7 @@ The Market Pallet emits the following events:
- `unsuccessful` - List of deal IDs with the corresponding errors
- `DealSlashed` - Is emitted when some deal expired
- `deal_id` - Deal ID that was slashed
- `DealTerminated` - Is emitted it indicates that the deal was voluntarily or involuntarily terminated.
- `DealTerminated` - A deal was voluntarily or involuntarily terminated.
- `deal_id` - Terminated deal ID
- `client` - SS58 address of the storage client
- `provider` - SS58 address of the storage provider
Expand Down
15 changes: 7 additions & 8 deletions docs/src/architecture/pallets/proofs.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,17 +13,16 @@
## Overview

The `Proofs Pallet` handles all the logic related to verifying [PoRep](../../glossary.md#porep) and [PoSt](../../glossary.md#post) proofs on-chain.
It's called by [`Storage Provider Pallet`](./storage-provider.md) when verifying proofs during the extrinsics: [`prove_commit_sectors`](./storage-provider.md#prove_commit_sectors)
It's called by [`Storage Provider Pallet`](./storage-provider.md) when verifying proofs during the extrinsics [`prove_commit_sectors`](./storage-provider.md#prove_commit_sectors)
and [`submit_windowed_post`](./storage-provider.md#submit_windowed_post). The Pallet **DOES NOT** expose any extrinsic for proofs verification, it only implements a trait that can be [coupled to other pallets](https://education.web3.foundation/docs/Substrate/section8/pallet-coupling).

To verify the proofs properly it needs to have the verifying keys parameters set for the sector size via [`set_porep_verifying_key`](#set_porep_verifying_key) and [`set_post_verifying_key](#set_post_verifying_key).

To verify the proofs properly it needs to have the verifying keys parameters set for the sector size via [`set_porep_verifying_key`](#set_porep_verifying_key) and [`set_post_verifying_key`](#set_post_verifying_key).

## Usage

This pallet can only be directly used via a trait `primitives_proofs::ProofVerification`. However, for the trait to work and not fail with `Error::MisingPoRepVerifyingKey`/`Error::MissingPoStVerifingKey`, first the verifying keys need to be set via extrinsics `set_porep_verifying_key`/`set_post_verifying_key`.
This pallet can only be directly used via the trait `primitives_proofs::ProofVerification`. However, for the trait to work and not fail with `Error::MisingPoRepVerifyingKey`/`Error::MissingPoStVerifingKey`, the verifying keys need to be set via extrinsics `set_porep_verifying_key`/`set_post_verifying_key`.

Ideally, users shouldn't worry about it, as it will be set by the governance during a trusted setup procedure and then Storage Providers will download proof generation parameters from somewhere. However, in the MVP phase, those keys need to be set with the extrinsics after starting a testnet.
Ideally, users shouldn't worry about it, as it will be set by the governance during a trusted setup procedure and then Storage Providers will download the proof generation parameters. However, in the MVP phase, those keys need to be set with the extrinsics after starting a testnet.

Verifying Keys are set for a Sector Size once and then shared across all proof verifications.
Currently, the network only supports 2KiB sector sizes, so parameters need to be generated and set for it.
Expand All @@ -32,7 +31,7 @@ Currently, the network only supports 2KiB sector sizes, so parameters need to be

### `set_porep_verifying_key`

Verifying Key is a set of shared parameters used for zk-SNARK proof verification. It can be generated via [`polka-storage-provider-client proofs porep-params`](../../storage-provider-cli/client/proofs.md#porep-params) command. The verifying key used in the verification must match proving parameters used in the proof generation.
Verifying Key is a set of shared parameters used for zk-SNARK proof verification. It can be generated via [`polka-storage-provider-client proofs porep-params`](../../storage-provider-cli/client/proofs.md#porep-params) command. The verifying key used in the verification must match the proving parameters used in the proof generation.

The extrinsic sets the verifying key received in the SCALE-encoded format and then uses it for all the subsequent verification.
Verifying Key is used to verify every PoRep proof across the network.
Expand All @@ -43,7 +42,7 @@ Verifying Key is used to verify every PoRep proof across the network.

#### <a class="header" id="set_porep_verifying_key.example" href="#set_porep_verifying_key.example">Example</a>

Setting a verifying key from an [^account] `//Alice` where proof is stored in `./2KiB.porep.vk.scale` file.
Setting a verifying key from the [^account] `//Alice` account where proof is stored in the `./2KiB.porep.vk.scale` file.

```bash
storagext-cli --sr25519-key "//Alice" proofs set-porep-verifying-key 2KiB.vk.scale
Expand All @@ -64,7 +63,7 @@ Verifying Key is used to verify every PoSt proof across the network.

#### <a class="header" id="set_post_verifying_key.example" href="#set_post_verifying_key.example">Example</a>

Setting a verifying key from an [^account] `//Alice` where proof is stored in `./2KiB.post.vk.scale` file.
Setting a verifying key from the [^account] `//Alice` account where proof is stored in the `./2KiB.post.vk.scale` file.

```bash
storagext-cli --sr25519-key "//Alice" proofs set-post-verifying-key 2KiB.vk.scale
Expand Down
24 changes: 13 additions & 11 deletions docs/src/architecture/pallets/randomness.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,24 +2,26 @@

## Table of Contents

- [Overview](#overview)
- [Usage](#usage)
- [Extrinsics](#extrinsics)
- [Events](#events)
- [Errors](#errors)
- [Constants](#constants)
- [Randomness Pallet](#randomness-pallet)
- [Table of Contents](#table-of-contents)
- [Overview](#overview)
- [Usage](#usage)
- [Extrinsics](#extrinsics)
- [Events](#events)
- [Errors](#errors)
- [Constants](#constants)

## Overview

The randomness pallet saves a random seed for each block when its finalized and allows to get this randomness later.
There is a limitation - the randomness is available only after 81st block of the chain, due to randomness predictability earlier.
Currently, the seeds are used for sealing pipeline's pre-commit and prove commit, so for generating a replica and proving a sector.
The randomness pallet saves a random seed for each block when it's finalized and allows retrieval of this randomness at a later time.
There is a limitation - the randomness is available only after the 81st block of the chain, due to randomness predictability earlier.
Currently, the seeds are used for the sealing pipeline's pre-commit and prove commit, in other words generating a replica and proving a sector.

## Usage

This pallet exposes the interface to get randomness on-chain for a certain block via a trait `primitives_proofs::Randomness`
This pallet exposes the interface to get randomness on-chain for a certain block via the trait `primitives_proofs::Randomness`
or chain state query `pallet_randomness:SeedsMap`.
Note that, you can only get a randomness for a `current_block - 1` and dependent on the configuration, the old randomness seed are being removed.
Note that, you can only get a randomness for the `current_block - 1` and depending on the configuration, the old randomness seed will be removed after the associated block has passed.

## Extrinsics

Expand Down
14 changes: 6 additions & 8 deletions docs/src/architecture/pallets/storage-provider.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@

The `Storage Provider Pallet` handles the creation of storage providers and facilitates storage providers and clients in creating storage deals.
Storage providers must provide the [Proof of Space-time (PoSt)](../../glossary.md#post) and the [Proof of Replication (PoRep)](../../glossary.md#porep)
to the `Storage Provider Pallet` to prevent the pallet from imposing penalties on storage providers through [slashing](#storage-fault-slashing).
to the `Storage Provider Pallet` to prevent the pallet from imposing penalties on storage providers through [slashing](../../glossary.md#slashing).

## Usage

Expand All @@ -38,8 +38,6 @@ the storage provider can declare sectors as faulty or recovered[^recovered].

[^recovered]: Recovered sectors still require being proven before they can become fully active again.

Substrate pallet hooks execute actions when certain conditions are met.

Substrate pallet hooks execute some actions when certain conditions are met.
We use these hooks — when a block finalizes — to check if storage providers are up to date with their proofs.
If a storage provider fails to submit proof on time, the Storage Provider pallet will signal the Market pallet to penalize the storage provider.
Expand Down Expand Up @@ -91,7 +89,7 @@ Sectors are not valid after pre-commit. The sectors need to be proven first.

#### <a class="header" id="pre_commit_sectors.example" href="#pre_commit_sectors.example">Example</a>

Storage provider `//Alice` pre-committing[^pre_commit_sectors] a sector number 1, with a single deal ID 0.
Storage provider `//Alice` pre-committing[^pre-committing] a sector number 1, with a single deal ID 0.

```bash
storagext-cli --sr25519-key "//Alice" storage-provider pre-commit @pre-commit-sector.json
Expand All @@ -110,7 +108,7 @@ Where `pre-commit-sector.json` is a file with contents similar to:
}
```

[^pre_commit_sectors]: Read more about the `pre-commit` command in [_Storagext CLI/Subcommand `storage-provider`/`pre-commit`_](../../storagext-cli/storage-provider.md#pre-commit)
[^pre-committing]: Read more about the `pre-commit` command in [_Storagext CLI/Subcommand `storage-provider`/`pre-commit`_](../../storagext-cli/storage-provider.md#pre-commit)

### `prove_commit_sectors`

Expand Down Expand Up @@ -275,8 +273,8 @@ Where the termination declarations contain:

| Name | Description | Type |
| ----------- | -------------------------------------------------------------------------- | ------------------------ |
| `deadline` | The deadline the termination is targeting | Positive integer. |
| `partition` | Partition index within the deadline containing the sector to be terminated | Positive integer. |
| `deadline` | The deadline the termination is targeting | Positive integer |
| `partition` | Partition index within the deadline containing the sector to be terminated | Positive integer |
| `sectors` | Sectors in the partition being terminated | Set of positive integers |

#### <a class="header" id="terminate_sectors.example" href="#terminate_sectors.example">Example</a>
Expand Down Expand Up @@ -405,7 +403,7 @@ The Storage Provider Pallet actions can fail with the following errors:
- `DeadlineErrorSectorsNotFound` - Emitted when trying to use sectors which haven't been prove committed yet.
- `DeadlineErrorSectorsNotFaulty` - Emitted when trying to recover non-faulty sectors.
- `DeadlineErrorCouldNotAssignSectorsToDeadlines` - Emitted when assigning sectors to deadlines fails.
- `DeadlineErrorFailedToUpdateFaultExpiration` - Emitted when trying to update fault expirations fails
- `DeadlineErrorFailedToUpdateFaultExpiration` - Emitted when trying to update fault expirations fails.
- `StorageProviderErrorMaxPreCommittedSectorExceeded` - Happens when an SP tries to pre-commit more sectors than SECTOR_MAX.
- `StorageProviderErrorSectorNotFound` - Happens when trying to access a sector that does not exist.
- `StorageProviderErrorSectorNumberInUse` - Happens when a sector number is already in use.
Expand Down
Loading