feat: forest-tool command for measuring RPC completeness and compat…

…ibility (#3616)

CHANGELOG.md

@@ -39,6 +39,8 @@
   server.
 - [#3613](https://github.com/ChainSafe/forest/pull/3613) Add `--expire-in`
   parameter to token commands.
+- [#3584](https://github.com/ChainSafe/forest/issues/3584) Add
+  `forest-tool api compare` command for testing RPC compatibility.
 
 ### Changed
 

Cargo.toml

@@ -166,6 +166,7 @@ static_assertions = "1.1.0"
 statrs = "0.16"
 strum = { version = "0.25", features = ["derive"] }
 strum_macros = "0.25"
+tabled = "0.14"
 tap = "1"
 tempfile = "3.4"
 thiserror = "1.0"

documentation/src/SUMMARY.md

@@ -30,5 +30,6 @@
   - [Devnet Notes](./developer_documentation/devnet_notes.md)
   - [Filecoin Archive](./developer_documentation/filecoin_archive.md)
   - [Archie and Fuzzy](./developer_documentation/archie_and_fuzzy.md)
+  - [RPC API Compatibility](./developer_documentation/rpc_api_compatibility.md)
   - [Notes and sketches](./developer_documentation/notes_and_sketches.md)
     - [ChainMuxer/TipsetProcessor state machine](./developer_documentation/chain_muxer_state_machine.md)

documentation/src/developer_documentation/rpc_api_compatibility.md

@@ -0,0 +1,88 @@
+# RPC Compatibility
+
+A running Lotus node can be accessed through an RPC interface. The RPC methods
+are listed here:
+
+- V0 methods (stable):
+  https://github.com/filecoin-project/lotus/blob/master/documentation/en/api-v0-methods.md
+- V1 methods (unstable):
+  https://github.com/filecoin-project/lotus/blob/master/documentation/en/api-v1-unstable-methods.md
+
+The current status of compatibility can be checked by comparing a running Forest
+node with a running Lotus node:
+
+1. Build Lotus with support for Calibnet and sync to HEAD.
+2. Run Forest against Calibnet and sync to HEAD.
+3. Run `forest-tool api compare`
+
+The output will look like this:
+
+| RPC Method                        | Forest              | Lotus         |
+| --------------------------------- | ------------------- | ------------- |
+| Filecoin.ChainGetBlock            | Valid               | Valid         |
+| Filecoin.ChainGetGenesis          | Valid               | Valid         |
+| Filecoin.ChainGetMessage (67)     | InternalServerError | Valid         |
+| Filecoin.ChainGetMessagesInTipset | MissingMethod       | Valid         |
+| Filecoin.ChainGetTipSetByHeight   | Valid               | Valid         |
+| Filecoin.ChainHead                | Valid               | Valid         |
+| Filecoin.ChainReadObj             | InvalidResponse     | Valid         |
+| Filecoin.Discover                 | MissingMethod       | Valid         |
+| Filecoin.MpoolPending             | Valid               | Valid         |
+| Filecoin.NetAddrsListen           | Valid               | Valid         |
+| Filecoin.NetInfo                  | Valid               | MissingMethod |
+| Filecoin.NetPeers                 | Valid               | Valid         |
+| Filecoin.Session                  | MissingMethod       | Valid         |
+| Filecoin.StartTime                | Valid               | Valid         |
+| Filecoin.StateGetActor            | InternalServerError | Valid         |
+| Filecoin.StateMinerPower (76)     | MissingMethod       | Valid         |
+| Filecoin.StateNetworkName         | Valid               | Valid         |
+| Filecoin.Version                  | Valid               | Valid         |
+
+If an entry for Lotus is not marked as `Valid`, this indicates that the Forest
+RPC client is buggy and incorrectly communicates with Lotus.
+
+## Limitations
+
+Forest aims at being a drop-in replacement for Lotus and have support for all of
+the RPC methods. Note, some methods (like `Filecoin.ChainHotGC`) are
+Lotus-specific and are meaningless in Forest. Such methods should be no-ops in
+Forest.
+
+Forest does not yet support mining and none of the mining-related RPC calls will
+be implemented in the foreseeable future.
+
+Forest does not support any v1 methods yets.
+
+## Gateway
+
+The `lotus-gateway` executable is a reverse-proxy that sanitizes RPC calls
+before they're forwarded to a Filecoin node. The `forest-tool api compare`
+command will fail if run against a gateway rather than directly against a node.
+This means API compatiblity testing has to be done with a local node rather than
+`api.node.glif.io`.
+
+## Use `mitmproxy`
+
+Inspecting RPC calls is best done with a reverse proxy. If Lotus listens to port
+1234 and Forest listens to port 2345, run the API compatibility tests through
+reverse proxies:
+
+1. `mitmproxy --node reverse:http://localhost:2345 --listen-port 8080`
+2. `mitmproxy --node reverse:http://localhost:1234 --listen-port 8081`
+3. `forest-tool api compare --forest /ip4/127.0.0.1/tcp/8080/http --lotus /ip4/127.0.0.1/tcp/8081/http`
+
+Request / Response pairs will show up in the `mitmproxy` windows.
+
+## Adding a new method
+
+Checklist for adding a new RPC method:
+
+1. Add method name in `src/rpc_api/mod.rs` and set the access level.
+2. Add request/response data types to `src/rpc_api/data_types.rs` as needed.
+3. Add `RpcRequest` in the appropriate file in `src/rpc_client/`.
+4. Test the method in `src/tool/subcommands/api_cmd.rs`. The method should show
+   up as `Valid` for Lotus and `MissingMethod` for Forest. Use `mitmproxy` to
+   debug.
+5. Implement Forest endpoint in `src/rpc/`, add it to the method list in
+   `src/rpc/mod.rs`
+6. Verify that the test from step 4 shows `Valid` for Forest.

src/lotus_json/mod.rs

@@ -122,6 +122,7 @@
 //! - use a derive macro for simple compound structs
 
 use derive_more::From;
+use fil_actor_interface::power::Claim;
 use serde::{de::DeserializeOwned, Deserialize, Deserializer, Serialize, Serializer};
 use serde_json::json;
 use std::{fmt::Display, str::FromStr};
@@ -327,7 +328,7 @@ where
 }
 
 /// A domain struct that is (de) serialized through its lotus JSON representation.
-#[derive(Serialize, Deserialize, From, Default)]
+#[derive(Debug, Serialize, Deserialize, From, Default)]
 #[serde(bound = "T: HasLotusJson + Clone")]
 pub struct LotusJson<T>(#[serde(with = "self")] pub T);
 
@@ -387,6 +388,34 @@ lotus_json_with_self!(
     bool,
 );
 
+#[derive(Default, Debug, Serialize, Deserialize)]
+#[serde(rename_all = "PascalCase")]
+pub struct ClaimLotusJson {
+    /// Sum of raw byte power for a miner's sectors.
+    pub raw_byte_power: LotusJson<num::BigInt>,
+    /// Sum of quality adjusted power for a miner's sectors.
+    pub quality_adj_power: LotusJson<num::BigInt>,
+}
+
+impl HasLotusJson for Claim {
+    type LotusJson = ClaimLotusJson;
+    fn snapshots() -> Vec<(serde_json::Value, Self)> {
+        vec![]
+    }
+    fn into_lotus_json(self) -> Self::LotusJson {
+        ClaimLotusJson {
+            raw_byte_power: LotusJson(self.raw_byte_power),
+            quality_adj_power: LotusJson(self.quality_adj_power),
+        }
+    }
+    fn from_lotus_json(lotus_json: Self::LotusJson) -> Self {
+        Claim {
+            raw_byte_power: lotus_json.raw_byte_power.into_inner(),
+            quality_adj_power: lotus_json.quality_adj_power.into_inner(),
+        }
+    }
+}
+
 impl<T: HasLotusJson> HasLotusJson for (T,) {
     type LotusJson = (T::LotusJson,);
     fn snapshots() -> Vec<(serde_json::Value, Self)> {

src/rpc_api/data_types.rs

@@ -12,6 +12,7 @@ use crate::key_management::KeyStore;
 pub use crate::libp2p::{Multiaddr, Protocol};
 use crate::libp2p::{Multihash, NetworkMessage};
 use crate::lotus_json::lotus_json_with_self;
+use crate::lotus_json::{HasLotusJson, LotusJson};
 use crate::message::signed_message::SignedMessage;
 use crate::message_pool::{MessagePool, MpoolRpcProvider};
 use crate::shim::executor::Receipt;
@@ -21,10 +22,13 @@ use ahash::HashSet;
 use chrono::Utc;
 use cid::Cid;
 use fil_actor_interface::market::{DealProposal, DealState};
+use fil_actor_interface::miner::MinerPower;
+use fil_actor_interface::power::Claim;
 use fvm_ipld_blockstore::Blockstore;
 use jsonrpc_v2::{MapRouter as JsonRpcMapRouter, Server as JsonRpcServer};
 use parking_lot::RwLock as SyncRwLock;
 use serde::{Deserialize, Serialize};
+use serde_json::Value;
 use tokio::sync::RwLock;
 
 /// This is where you store persistent data, or at least access to stateful
@@ -135,3 +139,99 @@ impl Version {
         Self((major as u32) << 16 | (minor as u32) << 8 | (patch as u32))
     }
 }
+
+#[derive(Debug, Serialize, Deserialize, Eq, PartialEq)]
+pub struct ApiMessage {
+    cid: Cid,
+    message: Message,
+}
+
+#[derive(Serialize, Deserialize)]
+#[serde(rename_all = "PascalCase")]
+pub struct ApiMessageLotusJson {
+    cid: LotusJson<Cid>,
+    message: LotusJson<Message>,
+}
+
+impl HasLotusJson for ApiMessage {
+    type LotusJson = ApiMessageLotusJson;
+    fn snapshots() -> Vec<(serde_json::Value, Self)> {
+        vec![]
+    }
+    fn into_lotus_json(self) -> Self::LotusJson {
+        ApiMessageLotusJson {
+            cid: LotusJson(self.cid),
+            message: LotusJson(self.message),
+        }
+    }
+    fn from_lotus_json(lotus_json: Self::LotusJson) -> Self {
+        ApiMessage {
+            cid: lotus_json.cid.into_inner(),
+            message: lotus_json.message.into_inner(),
+        }
+    }
+}
+
+#[derive(Serialize, Deserialize)]
+#[serde(rename_all = "PascalCase")]
+pub struct MinerPowerLotusJson {
+    miner_power: LotusJson<Claim>,
+    total_power: LotusJson<Claim>,
+    has_min_power: bool,
+}
+
+impl HasLotusJson for MinerPower {
+    type LotusJson = MinerPowerLotusJson;
+    fn snapshots() -> Vec<(serde_json::Value, Self)> {
+        vec![]
+    }
+    fn into_lotus_json(self) -> Self::LotusJson {
+        MinerPowerLotusJson {
+            miner_power: LotusJson(self.miner_power),
+            total_power: LotusJson(self.total_power),
+            has_min_power: self.has_min_power,
+        }
+    }
+    fn from_lotus_json(lotus_json: Self::LotusJson) -> Self {
+        MinerPower {
+            miner_power: lotus_json.miner_power.into_inner(),
+            total_power: lotus_json.total_power.into_inner(),
+            has_min_power: lotus_json.has_min_power,
+        }
+    }
+}
+
+#[derive(Debug, Serialize, Deserialize)]
+pub struct DiscoverResult {
+    info: DiscoverInfo,
+    methods: Vec<DiscoverMethod>,
+    openrpc: String,
+}
+
+#[derive(Debug, Serialize, Deserialize)]
+#[serde(rename_all = "camelCase")]
+pub struct DiscoverMethod {
+    deprecated: bool,
+    description: String,
+    external_docs: DiscoverDocs,
+    name: String,
+    param_structure: String,
+    params: Value,
+    // Missing 'result' field. Tracking issue:
+    // https://github.com/ChainSafe/forest/issues/3585
+    summary: String,
+}
+
+#[derive(Debug, Serialize, Deserialize)]
+pub struct DiscoverDocs {
+    description: String,
+    url: String,
+}
+
+#[derive(Debug, Serialize, Deserialize)]
+pub struct DiscoverInfo {
+    title: String,
+    version: String,
+}
+
+lotus_json_with_self!(DiscoverResult, DiscoverMethod, DiscoverDocs, DiscoverInfo);

src/rpc_api/mod.rs

@@ -189,6 +189,7 @@ pub mod chain_api {
     pub const CHAIN_GET_TIPSET: &str = "Filecoin.ChainGetTipSet";
     pub const CHAIN_SET_HEAD: &str = "Filecoin.ChainSetHead";
     pub const CHAIN_GET_MIN_BASE_FEE: &str = "Filecoin.ChainGetMinBaseFee";
+    pub const CHAIN_GET_MESSAGES_IN_TIPSET: &str = "Filecoin.ChainGetMessagesInTipset";
 }
 
 /// Message Pool API
@@ -232,6 +233,7 @@ pub mod state_api {
     pub const STATE_GET_RECEIPT: &str = "Filecoin.StateGetReceipt";
     pub const STATE_WAIT_MSG: &str = "Filecoin.StateWaitMsg";
     pub const STATE_FETCH_ROOT: &str = "Filecoin.StateFetchRoot";
+    pub const STATE_MINOR_POWER: &str = "Filecoin.StateMinerPower";
 }
 
 /// Gas API
@@ -247,6 +249,8 @@ pub mod common_api {
     pub const VERSION: &str = "Filecoin.Version";
     pub const SHUTDOWN: &str = "Filecoin.Shutdown";
     pub const START_TIME: &str = "Filecoin.StartTime";
+    pub const DISCOVER: &str = "Filecoin.Discover";
+    pub const SESSION: &str = "Filecoin.Session";
 }
 
 /// Net API

src/rpc_client/chain_ops.rs

@@ -1,6 +1,7 @@
 // Copyright 2019-2023 ChainSafe Systems
 // SPDX-License-Identifier: Apache-2.0, MIT
 
+use crate::rpc_api::data_types::ApiMessage;
 use crate::shim::message::Message;
 use crate::{
     blocks::{BlockHeader, Tipset, TipsetKeys},
@@ -101,4 +102,8 @@ impl ApiInfo {
     pub fn chain_get_min_base_fee_req(basefee_lookback: u32) -> RpcRequest<String> {
         RpcRequest::new(CHAIN_GET_MIN_BASE_FEE, (basefee_lookback,))
     }
+
+    pub fn chain_get_messages_in_tipset_req(tsk: TipsetKeys) -> RpcRequest<Vec<ApiMessage>> {
+        RpcRequest::new(CHAIN_GET_MESSAGES_IN_TIPSET, (tsk,))
+    }
 }