update schema

pinax-network · Aug 14, 2024 · 2cea9da · 2cea9da
1 parent c9d14e2
commit 2cea9da
Show file tree

Hide file tree

Showing 2 changed files with 44 additions and 40 deletions.
diff --git a/README.md b/README.md
@@ -3,40 +3,41 @@
 [![.github/workflows/bun-test.yml](https://github.com/pinax-network/erc20-token-api/actions/workflows/bun-test.yml/badge.svg)](https://github.com/pinax-network/erc20-token-api/actions/workflows/bun-test.yml)
 
 > Tokens information from all EVM blockchains, powered by [Substreams](https://substreams.streamingfast.io/)
+
 ## REST API
 
 ### Usage
 
-| Method | Path | Query parameters<br>(* = **Required**) | Description |
-| :---: | --- | --- | --- |
-| GET <br>`text/html` | `/` | - | [Swagger](https://swagger.io/) API playground |
-| GET <br>`application/json` | `/chains` | `limit`<br>`page` | Information about the chains and latest head block in the database |
-| GET <br>`application/json` | `/{chain}/balance` | `block_num`<br>`contract`<br>`account*`<br>`limit`<br>`page` | Balances of an account. |
-| GET <br>`application/json` | `/{chain}/holders` | **`contract*`**<br>`limit`<br>`page` | List of holders of a token |
-| GET <br>`application/json` | `/{chain}/supply` | `block_num`<br>`contract*`<br>`limit`<br>`page` | Total supply for a token |
-| GET <br>`application/json` | `/{chain}/tokens` |`contract`<br>`symbol`<br>`name`<br>`limit`<br>`page` | Get info about available tokens |
-| GET <br>`application/json` | `/{chain}/transfers` | `block_range`<br>`from`<br>`to`<br>`contract`<br>`limit`<br>`page` | All transfers related to a token |
-| GET <br>`application/json` | `/{chain}/transfers/{trx_id}` | `limit`<br>`page` | Specific transfer related to a token |
+|           Method           | Path                          | Query parameters<br>(\* = **Required**)                            | Description                                                        |
+| :------------------------: | ----------------------------- | ------------------------------------------------------------------ | ------------------------------------------------------------------ |
+|    GET <br>`text/html`     | `/`                           | -                                                                  | [Swagger](https://swagger.io/) API playground                      |
+| GET <br>`application/json` | `/chains`                     | `limit`<br>`page`                                                  | Information about the chains and latest head block in the database |
+| GET <br>`application/json` | `/{chain}/balance`            | `block_num`<br>`contract`<br>`account*`<br>`limit`<br>`page`       | Balances of an account.                                            |
+| GET <br>`application/json` | `/{chain}/holders`            | **`contract*`**<br>`limit`<br>`page`                               | List of holders of a token                                         |
+| GET <br>`application/json` | `/{chain}/supply`             | `block_num`<br>`contract*`<br>`limit`<br>`page`                    | Total supply for a token                                           |
+| GET <br>`application/json` | `/{chain}/tokens`             | `contract`<br>`symbol`<br>`name`<br>`limit`<br>`page`              | Get info about available tokens                                    |
+| GET <br>`application/json` | `/{chain}/transfers`          | `block_range`<br>`from`<br>`to`<br>`contract`<br>`limit`<br>`page` | All transfers related to a token                                   |
+| GET <br>`application/json` | `/{chain}/transfers/{trx_id}` | `limit`<br>`page`                                                  | Specific transfer related to a token                               |
 
 ### Docs
 
-| Method | Path | Description |
-| :---: | --- | --- |
+|           Method           | Path       | Description                                        |
+| :------------------------: | ---------- | -------------------------------------------------- |
 | GET <br>`application/json` | `/openapi` | [OpenAPI](https://www.openapis.org/) specification |
-| GET <br>`application/json` | `/version` | API version and Git short commit hash |
+| GET <br>`application/json` | `/version` | API version and Git short commit hash              |
 
 ### Monitoring
 
-| Method | Path | Description |
-| :---: | --- | --- |
-| GET <br>`text/plain` | `/health` | Checks database connection |
+|        Method        | Path       | Description                                  |
+| :------------------: | ---------- | -------------------------------------------- |
+| GET <br>`text/plain` | `/health`  | Checks database connection                   |
 | GET <br>`text/plain` | `/metrics` | [Prometheus](https://prometheus.io/) metrics |
 
 ## Requirements
 
-- [ClickHouse](clickhouse.com/), databases should follow a `{chain}_tokens_{version}` naming scheme. Database tables can be setup using the [`schema.sql`](./schema.sql) definitions created by the [`create_schema.sh`](./create_schema.sh) script.
-- A [Substream sink](https://substreams.streamingfast.io/reference-and-specs/glossary#sink) for loading data into ClickHouse. We recommend [Substreams Sink ClickHouse](https://github.com/pinax-network/substreams-sink-clickhouse/) or [Substreams Sink SQL](https://github.com/pinax-network/substreams-sink-sql). You should use the generated [`protobuf` files](tsp-output/@typespec/protobuf) to build your substream. This Token API makes use of the [`erc20-substreams`](https://github.com/pinax-network/erc20-substreams) substream.
-- [A Substreams API Token provider](https://pinax.network) to stream blockchains Data.
+-   [ClickHouse](clickhouse.com/), databases should follow a `{chain}_tokens_{version}` naming scheme. Database tables can be setup using the [`schema.sql`](./schema.sql) definitions created by the [`create_schema.sh`](./create_schema.sh) script.
+-   A [Substream sink](https://substreams.streamingfast.io/reference-and-specs/glossary#sink) for loading data into ClickHouse. We recommend [Substreams Sink ClickHouse](https://github.com/pinax-network/substreams-sink-clickhouse/) or [Substreams Sink SQL](https://github.com/pinax-network/substreams-sink-sql). You should use the generated [`protobuf` files](tsp-output/@typespec/protobuf) to build your substream. This Token API makes use of the [`erc20-substreams`](https://github.com/pinax-network/erc20-substreams) substream.
+-   [A Substreams API Token provider](https://pinax.network) to stream blockchains Data.
 
 ### API stack architecture
 
@@ -69,12 +70,11 @@ echo "CREATE DATABASE eth_tokens_v1" | clickhouse client -h <host> --port 9000 -
 4. Execute the schema
 
 ```console
-cat /tmp/schema.sql | clickhouse client -h <host> --port 9000 -d <database> -u <user> --password <password>
+clickhouse client -h <host> --port 9000 -d <database> -u <user> --password <password> --multiquery < ./schema.sql
 ```
 
 5. Run the [sink](https://github.com/pinax-network/substreams-sink-sql)
 
-
 ```console
 export SUBSTREAMS_TOKEN= "YOUR_SUBSTREAMS_TOKEN"
 ```
@@ -110,7 +110,6 @@ echo "CREATE DATABASE eth_tokens_v1 ON CLUSTER <cluster>" | clickhouse client -h
 ./create_schema.sh -o /tmp/schema.sql -c <cluster>
 ```
 
-
 ## [`Bun` Binary Releases](https://github.com/pinax-network/antelope-token-api/releases)
 
 > [!WARNING]
@@ -119,7 +118,7 @@ echo "CREATE DATABASE eth_tokens_v1 ON CLUSTER <cluster>" | clickhouse client -h
 ```console
 $ wget https://github.com/pinax-network/erc20-token-api/releases/download/v1.0.1/erc20-token-api
 $ chmod +x ./erc20-token-api
-$ ./erc20-token-api --help                                                                                                       
+$ ./erc20-token-api --help
 Usage: erc20-token-api [options]
 
 Token balances, supply and transfers from the Antelope blockchains
@@ -158,24 +157,28 @@ VERBOSE=true
 
 ## Docker environment
 
-- Pull from GitHub Container registry
+-   Pull from GitHub Container registry
 
 **For latest tagged release**
+
 ```bash
 docker pull ghcr.io/pinax-network/erc20-token-api:latest
 ```
 
 **For head of `main` branch**
+
 ```bash
 docker pull ghcr.io/pinax-network/erc20-token-api:develop
 ```
 
-- Build from source
+-   Build from source
+
 ```bash
 docker build -t erc20-token-api .
 ```
 
-- Run with `.env` file
+-   Run with `.env` file
+
 ```bash
 docker run -it --rm --env-file .env ghcr.io/pinax-network/erc20-token-api
 ```
@@ -194,6 +197,7 @@ $ bun dev
 ```
 
 **Tests**
+
 ```console
 $ bun lint
 $ bun test

diff --git a/create_schema.sh b/create_schema.sh
@@ -41,7 +41,7 @@ cat > $SCHEMA_FILE <<- EOM
 -- Meta tables to store Substreams information --
 -------------------------------------------------
 
-CREATE TABLE IF NOT EXISTS cursors ON CLUSTER tokenapis
+CREATE TABLE IF NOT EXISTS cursors 
 (
     id        String,
     cursor    String,
@@ -58,7 +58,7 @@ CREATE TABLE IF NOT EXISTS cursors ON CLUSTER tokenapis
 -- -- Table for all balance changes event --
 -------------------------------------------------
 
-CREATE TABLE IF NOT EXISTS balance_changes ON CLUSTER tokenapis  (
+CREATE TABLE IF NOT EXISTS balance_changes   (
     "id"            String,
     timestamp       DateTime64(3, 'UTC'),
     contract        FixedString(40),
@@ -78,7 +78,7 @@ ORDER BY (id,timestamp, block_num);
 -- -- MV for historical balance changes event order by contract address   --
 ------------------------------------------------------------------------------
 
-CREATE MATERIALIZED VIEW balance_changes_contract_historical_mv  ON CLUSTER tokenapis
+CREATE MATERIALIZED VIEW balance_changes_contract_historical_mv  
 ENGINE = ReplicatedReplacingMergeTree()
 ORDER BY (contract, owner,block_num)
 POPULATE
@@ -87,7 +87,7 @@ AS SELECT * FROM balance_changes;
 ------------------------------------------------------------------------------
 -- -- MV for historical balance changes event order by account address   --
 ------------------------------------------------------------------------------
-CREATE MATERIALIZED VIEW balance_changes_account_historical_mv ON CLUSTER tokenapis
+CREATE MATERIALIZED VIEW balance_changes_account_historical_mv 
 ENGINE = ReplicatedReplacingMergeTree()
 ORDER BY (owner, contract,block_num)
 POPULATE
@@ -97,7 +97,7 @@ AS SELECT * FROM balance_changes;
 -------------------------------------------
 -- -- MV for latest token_holders   --
 -------------------------------------------
-CREATE TABLE IF NOT EXISTS token_holders ON CLUSTER tokenapis
+CREATE TABLE IF NOT EXISTS token_holders 
 (
     account              FixedString(40),
     contract             String,
@@ -110,7 +110,7 @@ CREATE TABLE IF NOT EXISTS token_holders ON CLUSTER tokenapis
         PRIMARY KEY (contract,account)
         ORDER BY (contract, account);
 
-CREATE MATERIALIZED VIEW token_holders_mv ON CLUSTER tokenapis
+CREATE MATERIALIZED VIEW token_holders_mv 
     TO token_holders
 AS
 SELECT owner as account,
@@ -126,7 +126,7 @@ FROM balance_changes;
 -------------------------------------------
 --  MV for account balances   --
 -------------------------------------------
-CREATE TABLE IF NOT EXISTS account_balances ON CLUSTER tokenapis
+CREATE TABLE IF NOT EXISTS account_balances 
 (
     account              FixedString(40),
     contract             String,
@@ -139,7 +139,7 @@ CREATE TABLE IF NOT EXISTS account_balances ON CLUSTER tokenapis
         PRIMARY KEY (account,contract)
         ORDER BY (account,contract);
 
-CREATE MATERIALIZED VIEW account_balances_mv ON CLUSTER tokenapis
+CREATE MATERIALIZED VIEW account_balances_mv 
     TO account_balances
 AS
 SELECT owner as account,
@@ -154,7 +154,7 @@ FROM balance_changes;
 -------------------------------------------------
 --  Table for all token information --
 -------------------------------------------------
-CREATE TABLE IF NOT EXISTS contracts ON CLUSTER tokenapis  (
+CREATE TABLE IF NOT EXISTS contracts   (
     contract FixedString(40),
     name        String,
     symbol      String,
@@ -171,7 +171,7 @@ ORDER BY (contract);
 -------------------------------------------------
 --  Table for  token supply  --
 -------------------------------------------------
-CREATE TABLE IF NOT EXISTS supply ON CLUSTER tokenapis  (
+CREATE TABLE IF NOT EXISTS supply   (
     contract FixedString(40),
     supply       UInt256,
     block_num    UInt32(),
@@ -184,7 +184,7 @@ ORDER BY (contract,block_num);
 -------------------------------------------------
 --  table for all transfers events  --
 -------------------------------------------------
-CREATE TABLE IF NOT EXISTS transfers ON CLUSTER tokenapis (
+CREATE TABLE IF NOT EXISTS transfers  (
     id String,
     contract FixedString(40),
     `from` String,
@@ -203,7 +203,7 @@ ORDER BY (id, tx_id, block_num, timestamp);
 -------------------------------------------------
 --  MV for historical transfers events by contract address --
 -------------------------------------------------
-CREATE MATERIALIZED VIEW transfers_contract_historical_mv ON CLUSTER tokenapis
+CREATE MATERIALIZED VIEW transfers_contract_historical_mv 
 ENGINE = ReplicatedReplacingMergeTree()
 ORDER BY (contract, `from`,`to`,block_num)
 POPULATE
@@ -212,7 +212,7 @@ AS SELECT * FROM transfers;
 -------------------------------------------------
 --  MV for historical transfers events by from address --
 -------------------------------------------------
-CREATE MATERIALIZED VIEW transfers_from_historical_mv ON CLUSTER tokenapis
+CREATE MATERIALIZED VIEW transfers_from_historical_mv 
 ENGINE = ReplicatedReplacingMergeTree()
 ORDER BY (`from`, contract,block_num)
 POPULATE
@@ -221,7 +221,7 @@ AS SELECT * FROM transfers;
 -------------------------------------------------
 --  MV for historical transfers events by to address --
 -------------------------------------------------
-CREATE MATERIALIZED VIEW transfers_to_historical_mv ON CLUSTER tokenapis
+CREATE MATERIALIZED VIEW transfers_to_historical_mv 
 ENGINE = ReplicatedReplacingMergeTree()
 ORDER BY (`to`, contract,block_num)
 POPULATE