diff --git a/README.md b/README.md index d2c6a45..5ac854f 100644 --- a/README.md +++ b/README.md @@ -3,40 +3,41 @@ [![.github/workflows/bun-test.yml](https://github.com/pinax-network/erc20-token-api/actions/workflows/bun-test.yml/badge.svg)](https://github.com/pinax-network/erc20-token-api/actions/workflows/bun-test.yml) > Tokens information from all EVM blockchains, powered by [Substreams](https://substreams.streamingfast.io/) + ## REST API ### Usage -| Method | Path | Query parameters
(* = **Required**) | Description | -| :---: | --- | --- | --- | -| GET
`text/html` | `/` | - | [Swagger](https://swagger.io/) API playground | -| GET
`application/json` | `/chains` | `limit`
`page` | Information about the chains and latest head block in the database | -| GET
`application/json` | `/{chain}/balance` | `block_num`
`contract`
`account*`
`limit`
`page` | Balances of an account. | -| GET
`application/json` | `/{chain}/holders` | **`contract*`**
`limit`
`page` | List of holders of a token | -| GET
`application/json` | `/{chain}/supply` | `block_num`
`contract*`
`limit`
`page` | Total supply for a token | -| GET
`application/json` | `/{chain}/tokens` |`contract`
`symbol`
`name`
`limit`
`page` | Get info about available tokens | -| GET
`application/json` | `/{chain}/transfers` | `block_range`
`from`
`to`
`contract`
`limit`
`page` | All transfers related to a token | -| GET
`application/json` | `/{chain}/transfers/{trx_id}` | `limit`
`page` | Specific transfer related to a token | +| Method | Path | Query parameters
(\* = **Required**) | Description | +| :------------------------: | ----------------------------- | ------------------------------------------------------------------ | ------------------------------------------------------------------ | +| GET
`text/html` | `/` | - | [Swagger](https://swagger.io/) API playground | +| GET
`application/json` | `/chains` | `limit`
`page` | Information about the chains and latest head block in the database | +| GET
`application/json` | `/{chain}/balance` | `block_num`
`contract`
`account*`
`limit`
`page` | Balances of an account. | +| GET
`application/json` | `/{chain}/holders` | **`contract*`**
`limit`
`page` | List of holders of a token | +| GET
`application/json` | `/{chain}/supply` | `block_num`
`contract*`
`limit`
`page` | Total supply for a token | +| GET
`application/json` | `/{chain}/tokens` | `contract`
`symbol`
`name`
`limit`
`page` | Get info about available tokens | +| GET
`application/json` | `/{chain}/transfers` | `block_range`
`from`
`to`
`contract`
`limit`
`page` | All transfers related to a token | +| GET
`application/json` | `/{chain}/transfers/{trx_id}` | `limit`
`page` | Specific transfer related to a token | ### Docs -| Method | Path | Description | -| :---: | --- | --- | +| Method | Path | Description | +| :------------------------: | ---------- | -------------------------------------------------- | | GET
`application/json` | `/openapi` | [OpenAPI](https://www.openapis.org/) specification | -| GET
`application/json` | `/version` | API version and Git short commit hash | +| GET
`application/json` | `/version` | API version and Git short commit hash | ### Monitoring -| Method | Path | Description | -| :---: | --- | --- | -| GET
`text/plain` | `/health` | Checks database connection | +| Method | Path | Description | +| :------------------: | ---------- | -------------------------------------------- | +| GET
`text/plain` | `/health` | Checks database connection | | GET
`text/plain` | `/metrics` | [Prometheus](https://prometheus.io/) metrics | ## Requirements -- [ClickHouse](clickhouse.com/), databases should follow a `{chain}_tokens_{version}` naming scheme. Database tables can be setup using the [`schema.sql`](./schema.sql) definitions created by the [`create_schema.sh`](./create_schema.sh) script. -- A [Substream sink](https://substreams.streamingfast.io/reference-and-specs/glossary#sink) for loading data into ClickHouse. We recommend [Substreams Sink ClickHouse](https://github.com/pinax-network/substreams-sink-clickhouse/) or [Substreams Sink SQL](https://github.com/pinax-network/substreams-sink-sql). You should use the generated [`protobuf` files](tsp-output/@typespec/protobuf) to build your substream. This Token API makes use of the [`erc20-substreams`](https://github.com/pinax-network/erc20-substreams) substream. -- [A Substreams API Token provider](https://pinax.network) to stream blockchains Data. +- [ClickHouse](clickhouse.com/), databases should follow a `{chain}_tokens_{version}` naming scheme. Database tables can be setup using the [`schema.sql`](./schema.sql) definitions created by the [`create_schema.sh`](./create_schema.sh) script. +- A [Substream sink](https://substreams.streamingfast.io/reference-and-specs/glossary#sink) for loading data into ClickHouse. We recommend [Substreams Sink ClickHouse](https://github.com/pinax-network/substreams-sink-clickhouse/) or [Substreams Sink SQL](https://github.com/pinax-network/substreams-sink-sql). You should use the generated [`protobuf` files](tsp-output/@typespec/protobuf) to build your substream. This Token API makes use of the [`erc20-substreams`](https://github.com/pinax-network/erc20-substreams) substream. +- [A Substreams API Token provider](https://pinax.network) to stream blockchains Data. ### API stack architecture @@ -69,12 +70,11 @@ echo "CREATE DATABASE eth_tokens_v1" | clickhouse client -h --port 9000 - 4. Execute the schema ```console -cat /tmp/schema.sql | clickhouse client -h --port 9000 -d -u --password +clickhouse client -h --port 9000 -d -u --password --multiquery < ./schema.sql ``` 5. Run the [sink](https://github.com/pinax-network/substreams-sink-sql) - ```console export SUBSTREAMS_TOKEN= "YOUR_SUBSTREAMS_TOKEN" ``` @@ -110,7 +110,6 @@ echo "CREATE DATABASE eth_tokens_v1 ON CLUSTER " | clickhouse client -h ./create_schema.sh -o /tmp/schema.sql -c ``` - ## [`Bun` Binary Releases](https://github.com/pinax-network/antelope-token-api/releases) > [!WARNING] @@ -119,7 +118,7 @@ echo "CREATE DATABASE eth_tokens_v1 ON CLUSTER " | clickhouse client -h ```console $ wget https://github.com/pinax-network/erc20-token-api/releases/download/v1.0.1/erc20-token-api $ chmod +x ./erc20-token-api -$ ./erc20-token-api --help +$ ./erc20-token-api --help Usage: erc20-token-api [options] Token balances, supply and transfers from the Antelope blockchains @@ -158,24 +157,28 @@ VERBOSE=true ## Docker environment -- Pull from GitHub Container registry +- Pull from GitHub Container registry **For latest tagged release** + ```bash docker pull ghcr.io/pinax-network/erc20-token-api:latest ``` **For head of `main` branch** + ```bash docker pull ghcr.io/pinax-network/erc20-token-api:develop ``` -- Build from source +- Build from source + ```bash docker build -t erc20-token-api . ``` -- Run with `.env` file +- Run with `.env` file + ```bash docker run -it --rm --env-file .env ghcr.io/pinax-network/erc20-token-api ``` @@ -194,6 +197,7 @@ $ bun dev ``` **Tests** + ```console $ bun lint $ bun test diff --git a/create_schema.sh b/create_schema.sh index be856c6..bc4cb01 100644 --- a/create_schema.sh +++ b/create_schema.sh @@ -41,7 +41,7 @@ cat > $SCHEMA_FILE <<- EOM -- Meta tables to store Substreams information -- ------------------------------------------------- -CREATE TABLE IF NOT EXISTS cursors ON CLUSTER tokenapis +CREATE TABLE IF NOT EXISTS cursors ( id String, cursor String, @@ -58,7 +58,7 @@ CREATE TABLE IF NOT EXISTS cursors ON CLUSTER tokenapis -- -- Table for all balance changes event -- ------------------------------------------------- -CREATE TABLE IF NOT EXISTS balance_changes ON CLUSTER tokenapis ( +CREATE TABLE IF NOT EXISTS balance_changes ( "id" String, timestamp DateTime64(3, 'UTC'), contract FixedString(40), @@ -78,7 +78,7 @@ ORDER BY (id,timestamp, block_num); -- -- MV for historical balance changes event order by contract address -- ------------------------------------------------------------------------------ -CREATE MATERIALIZED VIEW balance_changes_contract_historical_mv ON CLUSTER tokenapis +CREATE MATERIALIZED VIEW balance_changes_contract_historical_mv ENGINE = ReplicatedReplacingMergeTree() ORDER BY (contract, owner,block_num) POPULATE @@ -87,7 +87,7 @@ AS SELECT * FROM balance_changes; ------------------------------------------------------------------------------ -- -- MV for historical balance changes event order by account address -- ------------------------------------------------------------------------------ -CREATE MATERIALIZED VIEW balance_changes_account_historical_mv ON CLUSTER tokenapis +CREATE MATERIALIZED VIEW balance_changes_account_historical_mv ENGINE = ReplicatedReplacingMergeTree() ORDER BY (owner, contract,block_num) POPULATE @@ -97,7 +97,7 @@ AS SELECT * FROM balance_changes; ------------------------------------------- -- -- MV for latest token_holders -- ------------------------------------------- -CREATE TABLE IF NOT EXISTS token_holders ON CLUSTER tokenapis +CREATE TABLE IF NOT EXISTS token_holders ( account FixedString(40), contract String, @@ -110,7 +110,7 @@ CREATE TABLE IF NOT EXISTS token_holders ON CLUSTER tokenapis PRIMARY KEY (contract,account) ORDER BY (contract, account); -CREATE MATERIALIZED VIEW token_holders_mv ON CLUSTER tokenapis +CREATE MATERIALIZED VIEW token_holders_mv TO token_holders AS SELECT owner as account, @@ -126,7 +126,7 @@ FROM balance_changes; ------------------------------------------- -- MV for account balances -- ------------------------------------------- -CREATE TABLE IF NOT EXISTS account_balances ON CLUSTER tokenapis +CREATE TABLE IF NOT EXISTS account_balances ( account FixedString(40), contract String, @@ -139,7 +139,7 @@ CREATE TABLE IF NOT EXISTS account_balances ON CLUSTER tokenapis PRIMARY KEY (account,contract) ORDER BY (account,contract); -CREATE MATERIALIZED VIEW account_balances_mv ON CLUSTER tokenapis +CREATE MATERIALIZED VIEW account_balances_mv TO account_balances AS SELECT owner as account, @@ -154,7 +154,7 @@ FROM balance_changes; ------------------------------------------------- -- Table for all token information -- ------------------------------------------------- -CREATE TABLE IF NOT EXISTS contracts ON CLUSTER tokenapis ( +CREATE TABLE IF NOT EXISTS contracts ( contract FixedString(40), name String, symbol String, @@ -171,7 +171,7 @@ ORDER BY (contract); ------------------------------------------------- -- Table for token supply -- ------------------------------------------------- -CREATE TABLE IF NOT EXISTS supply ON CLUSTER tokenapis ( +CREATE TABLE IF NOT EXISTS supply ( contract FixedString(40), supply UInt256, block_num UInt32(), @@ -184,7 +184,7 @@ ORDER BY (contract,block_num); ------------------------------------------------- -- table for all transfers events -- ------------------------------------------------- -CREATE TABLE IF NOT EXISTS transfers ON CLUSTER tokenapis ( +CREATE TABLE IF NOT EXISTS transfers ( id String, contract FixedString(40), `from` String, @@ -203,7 +203,7 @@ ORDER BY (id, tx_id, block_num, timestamp); ------------------------------------------------- -- MV for historical transfers events by contract address -- ------------------------------------------------- -CREATE MATERIALIZED VIEW transfers_contract_historical_mv ON CLUSTER tokenapis +CREATE MATERIALIZED VIEW transfers_contract_historical_mv ENGINE = ReplicatedReplacingMergeTree() ORDER BY (contract, `from`,`to`,block_num) POPULATE @@ -212,7 +212,7 @@ AS SELECT * FROM transfers; ------------------------------------------------- -- MV for historical transfers events by from address -- ------------------------------------------------- -CREATE MATERIALIZED VIEW transfers_from_historical_mv ON CLUSTER tokenapis +CREATE MATERIALIZED VIEW transfers_from_historical_mv ENGINE = ReplicatedReplacingMergeTree() ORDER BY (`from`, contract,block_num) POPULATE @@ -221,7 +221,7 @@ AS SELECT * FROM transfers; ------------------------------------------------- -- MV for historical transfers events by to address -- ------------------------------------------------- -CREATE MATERIALIZED VIEW transfers_to_historical_mv ON CLUSTER tokenapis +CREATE MATERIALIZED VIEW transfers_to_historical_mv ENGINE = ReplicatedReplacingMergeTree() ORDER BY (`to`, contract,block_num) POPULATE