First go at the Fluffy book using mkdocs as for nimbus.guide (#1842)

* First go at the Fluffy book using mkdocs as for nimbus.guide

* reorganize a little the index page

* Add small paragraph on the what in index page

.github/workflows/fluffy_docs.yml

@@ -0,0 +1,32 @@
+name: fluffy docs CI
+
+on:
+  push:
+    branches:
+      - master
+    paths:
+      - '.github/workflows/fluffy_docs.yml'
+      - 'fluffy/docs/**'
+
+permissions:
+  contents: write
+jobs:
+  deploy:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+      - uses: actions/setup-python@v4
+        with:
+          python-version: 3.x
+      - run: echo "cache_id=$(date --utc '+%V')" >> $GITHUB_ENV
+      - uses: actions/cache@v3
+        with:
+          key: mkdocs-material-${{ env.cache_id }}
+          path: .cache
+          restore-keys: |
+            mkdocs-material-
+      - run: pip install mkdocs-material
+      - name: Run mkdocs github deploy
+        working-directory: ./fluffy/docs/the_fluffy_book/
+        run: |
+          mkdocs gh-deploy --force

fluffy/README.md

@@ -121,26 +121,6 @@ Find more details on the usage and workings of the local testnet script
 Follow the steps outlined [here](../README.md#windows) to build fluffy on Windows.
 
 
-## Development tools and documentation
-
-The fluffy directory also holds several tools to help development of the Portal
-networks.
-
-Command to build the tools:
-
-```bash
-make fluffy-tools -j6
-```
-
-Additional documention on the tools or on what you can use them for:
-
-- [eth_data_exporter](./docs/eth_data_exporter.md): tool to extract content from
-EL or CL and prepare it as Portal content and content keys.
-- [Content seeding](./docs/content_seeding.md): Documentation on how to retrieve & generate history data and how to seed it into the network
-- [Manual protocol interop testing](./docs/protocol_interop.md): commands on how to manually test the discv5 and Portal protocol request and responses
-- [Local testnet script](./docs/local_testnet.md): Documentation on the local testnet script and how to use it
-
-
 ## The basics for developers
 
 When working on this repository, you can run the `env.sh` script to run a

fluffy/docs/the_fluffy_book/docs/beacon-content-bridging.md

@@ -0,0 +1,25 @@
+# Bridging content into the Portal beacon network
+
+## Seeding from content bridges
+
+Run a Fluffy node with the JSON-RPC API enabled.
+
+```bash
+./build/fluffy --rpc
+```
+
+Build & run the `beacon_chain_bridge`:
+```bash
+make beacon_chain_bridge
+
+TRUSTED_BLOCK_ROOT=0x1234567890123456789012345678901234567890123456789012345678901234 # Replace with trusted block root.
+# --rest-url = access to beacon node API, default http://127.0.0.1:5052
+./build/beacon_chain_bridge --trusted-block-root:${TRUSTED_BLOCK_ROOT} --rest-url:http://127.0.0.1:5052
+```
+
+The `beacon_chain_bridge` will connect to Fluffy node over the JSON-RPC
+interface and start gossiping an `LightClientBootstrap` for
+given trusted block root and gossip backfill `LightClientUpdate`s.
+
+Next, it will gossip a new `LightClientOptimisticUpdate`,
+`LightClientFinalityUpdate` and `LightClientUpdate` as they become available.

fluffy/docs/eth_data_exporter.md → fluffy/docs/the_fluffy_book/docs/eth-data-exporter.md

@@ -1,4 +1,6 @@
-# eth_data_exporter
+# Exporting Ethereum content for Portal
+
+## eth_data_exporter
 
 The `eth_data_exporter` is a tool to extract content from Ethereum EL or CL and
 prepare it as Portal content and content keys.
@@ -9,12 +11,14 @@ Currently the `history` and the `beacon` networks are supported.
 Example commands:
 
 ```bash
+# Build the tool
+make eth_data_exporter
 # See the different commands and options
 ./build/eth_data_exporter --help
 ```
 
 ```bash
-# Request of Beacon Chain Light Client Updates and export into the Portal
+# Request of `BeaconLightClientUpdate`s and export into the Portal
 # network supported format
 ./build/eth_data_exporter beacon exportLCUpdates --rest-url:http://testing.mainnet.beacon-api.nimbus.team --start-period:816 --count:4
 ```

fluffy/docs/content_seeding.md → fluffy/docs/the_fluffy_book/docs/history-content-bridging.md

@@ -1,8 +1,38 @@
-# Seeding data into the Portal history network
+# Bridging content: Portal history network
 
-## Seeding from locally stored history data
+## From content bridges
 
-### Building and seeding epoch accumulators into the Portal history network
+### Seeding post-merge block data through the `beacon_lc_bridge`
+
+Run a Fluffy node with the JSON-RPC API enabled.
+
+```bash
+./build/fluffy --rpc
+```
+
+Build & run the `beacon_lc_bridge`:
+```bash
+make beacon_lc_bridge
+
+TRUSTED_BLOCK_ROOT=0x1234567890123456789012345678901234567890123456789012345678901234 # Replace with trusted block root.
+./build/beacon_lc_bridge --trusted-block-root=${TRUSTED_BLOCK_ROOT}
+```
+
+The `beacon_lc_bridge` will start with the consensus light client sync follow
+beacon block gossip. Once it is synced, the execution payload of new beacon
+blocks will be extracted and injected in the Portal network as execution headers
+and blocks.
+
+> Note: The execution headers will come without a proof for now.
+
+The injection into the Portal network is done via the
+`portal_historyGossip` JSON-RPC endpoint of the running Fluffy node.
+
+> Note: Backfilling of block bodies and headers is not yet supported.
+
+## From locally stored block data
+
+### Building and seeding epoch accumulators
 
 #### Step 1: Building the epoch accumulators
 1. Set-up access to an Ethereum JSON-RPC endpoint (e.g. local geth instance)
@@ -12,7 +42,7 @@ that can serve the data.
 *.e2s files arranged per epoch (8192 blocks):
 
 ```bash
-make fluffy-tools
+make eth_data_exporter
 
 ./build/eth_data_exporter history exportEpochHeaders --data-dir:"./user_data_dir/"
 ```
@@ -31,7 +61,7 @@ Run Fluffy and trigger the propagation of data with the
 `portal_history_propagateEpochAccumulators` JSON-RPC API call:
 
 ```bash
-./build/fluffy --rpc --table-ip-limit:1024 --bucket-ip-limit:24
+./build/fluffy --rpc
 
 # From another terminal
 curl -s -X POST -H 'Content-Type: application/json' -d '{"jsonrpc":"2.0","id":"1","method":"portal_history_propagateEpochAccumulators","params":["./user_data_dir/"]}' http://localhost:8545 | jq
@@ -44,15 +74,16 @@ accumulators are available on the history network:
 
 Make sure you still have a fluffy instance running, if not run:
 ```bash
-./build/fluffy --rpc --table-ip-limit:1024 --bucket-ip-limit:24
+./build/fluffy --rpc
 ```
 
 Run the `content_verifier` tool and see if all epoch accumulators are found:
 ```bash
+make content_verifier
 ./build/content_verifier
 ```
 
-### Downloading & seeding block data into the Portal network
+### Downloading & seeding block data
 
 1. Set-up access to an Ethereum JSON-RPC endpoint (e.g. local geth instance)
 that can serve the data.
@@ -61,7 +92,7 @@ JSON-RPC endpoint into the format which is suitable for reading data into
 Fluffy client and propagating into the network:
 
 ```bash
-make fluffy-tools
+make eth_data_exporter
 
 ./build/eth_data_exporter history exportBlockData--initial-block:1 --end-block:10 --data-dir:"/user_data_dir/"
 ```
@@ -73,38 +104,8 @@ This will store blocks 1 to 10 into a json file located at
 `portal_history_propagate` JSON-RPC API call:
 
 ```bash
-./build/fluffy --table-ip-limit:1024 --bucket-ip-limit:24 --log-level:info --rpc
+./build/fluffy --rpc
 
 # From another shell
 curl -s -X POST -H 'Content-Type: application/json' -d '{"jsonrpc":"2.0","id":"1","method":"portal_history_propagate","params":["./user_data_dir/eth-history-data.json"]}' http://localhost:8545 | jq
 ```
-
-## Seeding from content bridges
-
-### Seeding post-merge block headers and bodies through the beacon chain bridge
-
-Run a Fluffy node with the JSON-RPC API enabled.
-
-```bash
-./build/fluffy --rpc --table-ip-limit:1024 --bucket-ip-limit:24
-```
-
-Build & run the `beacon_lc_bridge`:
-```bash
-make beacon_lc_bridge
-
-TRUSTED_BLOCK_ROOT=0x1234567890123456789012345678901234567890123456789012345678901234 # Replace this
-./build/beacon_lc_bridge --trusted-block-root=${TRUSTED_BLOCK_ROOT}
-```
-
-The `beacon_lc_bridge` will start with the consensus light client sync follow
-beacon block gossip. Once it is synced, the execution payload of new beacon
-blocks will be extracted and injected in the Portal network as execution headers
-and blocks.
-
-> Note: The execution headers will come without a proof for now.
-
-The injection into the Portal network is done via the
-`portal_historyGossip` JSON-RPC endpoint of the running Fluffy node.
-
-> Note: Backfilling of block bodies and headers is not yet supported.

fluffy/docs/the_fluffy_book/docs/index.md

@@ -0,0 +1,42 @@
+# The Nimbus Fluffy Guide
+
+Fluffy is the Nimbus client implementation of the
+[Portal network specifications](https://github.com/ethereum/portal-network-specs/blob/master/portal-network.md).
+
+The Portal Network aims to deliver a reliable, sync-free, and decentralized
+access to the Ethereum blockchain. The network can be used by a light client to
+get access to Ethereum data and as such become a drop-in replacement for full
+nodes by providing that data through the existing
+[Ethereum JSON RPC Execution API](https://github.com/ethereum/execution-apis).
+
+This book describes how to build, run and monitor the Fluffy client, and how to
+use and test its currently implemented functionality.
+
+To quickly get your Fluffy node up and running, follow the quickstart page:
+
+  - [Quickstart for Linux / macOS users](./quick-start.md)
+  - [Quickstart for Windows users](./quick-start-windows.md)
+
+# Development status
+The Portal Network is a project still in research phase.
+This client is thus still experimental.
+
+However, the Portal history and Portal beacon sub-networks are already operational and can be tested on the public testnet or in a local testnet.
+
+## Get in touch
+
+Need help with anything?
+Join us on [Status](https://join.status.im/nimbus-general) and [Discord](https://discord.gg/9dWwPnG).
+
+## Donate
+
+If you'd like to contribute to Nimbus development:
+
+* Our donation address is [`0xDeb4A0e8d9a8dB30a9f53AF2dCc9Eb27060c6557`](https://etherscan.io/address/0xDeb4A0e8d9a8dB30a9f53AF2dCc9Eb27060c6557)
+* We're also listed on [GitCoin](https://gitcoin.co/grants/137/nimbus-2)
+
+## Disclaimer
+
+This documentation assumes Nimbus Fluffy is in its ideal state.
+The project is still under heavy development.
+Please submit a [Github issue](https://github.com/status-im/nimbus-eth1/issues) if you come across a problem.

fluffy/docs/protocol_interop.md → fluffy/docs/the_fluffy_book/docs/protocol-interop-testing.md

@@ -1,17 +1,18 @@
-# Testing Client Protocol Interoperability
-This document shows some commands that can be used to test the individual
-protocol messages per network (Discovery v5 and Portal networks).
+# Protocol Interoperability Testing
+This document shows a set of commands that can be used to test the individual
+protocol messages per network (Discovery v5 and Portal networks), e.g. to test
+client protocol interoperability.
 
-Two ways are explained, the first by keeping a node running and interacting
-with it through the JSON-RPC service. The second by running cli applications
+Two ways are explained, the first, by keeping a node running and interacting
+with it through the JSON-RPC service. The second, by running cli applications
 that attempt to send 1 specific message and then shutdown.
 
-The first is more powerful and complete, the second might be easier to do some
-quick testing.
+The first is more powerful and complete, the second one might be easier to do
+some quick testing.
 
-## Run Fluffy and test protocol messages launched via JSON-RPC API
+## Run Fluffy and test protocol messages via JSON-RPC API
 
-First build Fluffy as explained [here](../README.md#build-fluffy-client).
+First build Fluffy as explained [here](./quick-start.md#build-the-fluffy-client).
 
 Next run it with the JSON-RPC server enabled:
 ```bash
@@ -59,7 +60,7 @@ curl -s -X POST -H 'Content-Type: application/json' -d '{"jsonrpc":"2.0","id":"1
 > The `portal_state_` prefix can be replaced for testing other networks such as
 `portal_history_`.
 
-## Test Discovery and Portal Wire protocol messages through cli tools
+## Test Discovery and Portal Wire protocol messages with cli tools
 
 ### Testing Discovery v5 Layer: dcli
 
@@ -90,7 +91,7 @@ and ENR.
 
 ```bash
 # Build portalcli
-make fluffy-tools
+make portalcli
 ```
 
 With the `portalcli` tool you can test the individual Portal wire protocol
@@ -107,7 +108,7 @@ messages, e.g.:
 # Test Portal wire FindContent, should print the returned content
 ./build/portalcli findcontent enr:<base64 encoding of ENR>
 
-# Default the State network is tested, but you can provide another protocol id
+# Default the history network is tested, but you can provide another protocol id
 ./build/portalcli ping enr:<base64 encoding of ENR> --protocol-id:0x500B
 ```
 

fluffy/docs/the_fluffy_book/docs/quick-start-docker.md

@@ -0,0 +1,10 @@
+# Quick start - Docker
+
+This page takes you through the steps of getting the Fluffy Portal node running
+on the public network by use of the [public Docker image](https://hub.docker.com/r/statusim/nimbus-fluffy/tags).
+
+The docker image gets currently rebuild from latest master every night.
+
+## Steps
+
+To be added.