Skip to content

Commit

Permalink
update subql config and readme (#1057)
Browse files Browse the repository at this point in the history 
* update subql config and readme

* update

* add back old configs for backward compatibilty

* fix

* polish
  • Loading branch information
@shunjizhan
shunjizhan authored Nov 14, 2024
1 parent d2ca281 commit 5d78934
Show file tree
Hide file tree
Showing 6 changed files with 109 additions and 175 deletions.
224 changes: 90 additions & 134 deletions packages/evm-subql/README.md
View file
Original file line number Diff line number Diff line change
@@ -1,116 +1,30 @@
# @acala-network/evm-subql

Subquery services that index and query Acala EVM+ transactions and logs.

## Run

### Prepare

- install dependencies

```
yarn
```

- generate the required types from the GraphQL schema, and build code

```shell
yarn build
```

### Run all of the services using Docker

This will run all 4 services together:
1. A Mandala node with `--instant-sealing` in development mode
2. A Postgres database
3. A SubQl indexer that indexes data into Postgres
4. A SubQl query that wraps the data with GraphQl interface

```shell
docker-compose down -v # clean docker volume (optional)

docker-compose up
```

Please note that the indexer won't start until we [feed some transactions to the node](#feed-evm-transactions-to-node) if the node is running with `--instant-sealing`

### Run each service in the CLI seperately

Optionally, if you don't want to use the Docker, you can also run each service separately in the CLI ([official documentation](https://academy.subquery.network/run_publish/run.html#running-an-indexer-subql-node)).

NOTE: using CLI is for **local testing** only, so you can get familiar with each service. [For production](#for-production) please use docker setup instead.

1. Install SubQl globally

```shell
npm i -g @subql/[email protected] @subql/[email protected]
```

2. Run an [Acala](https://github.com/AcalaNetwork/Acala) node locally and listen to port number `9944` (in the first terminal)

If you already have a node running elsewhere, you can skip this step.

```shell
docker run -it --rm -p 9944:9944 ghcr.io/acalanetwork/mandala-node:sha-16f147e --dev --rpc-external --rpc-cors=all --rpc-methods=unsafe -levm=debug --pruning=archive --instant-sealing
```

3. Run a Postgres service and listen to port number 5432 (in the second terminal)

```shell
docker run -it -p 5432:5432 -e POSTGRES_PASSWORD=postgres postgres:12-alpine
```

4. [Feed some transactions to node](#feeding-evm-transactions-to-the-node) if the node is running with `--instant-sealing`

5. Run a subquery indexer (in the third terminal)

```shell
export DB_USER=postgres
export DB_PASS=postgres
export DB_DATABASE=postgres
export DB_HOST=localhost
export DB_PORT=5432

yarn index
```

6. Run the Query service (in the fourth terminal)

## Run with Docker
```shell
export DB_USER=postgres
export DB_PASS=postgres
export DB_DATABASE=postgres
export DB_HOST=localhost
export DB_PORT=5432

yarn query
docker compose down -v # clean docker volume
docker compose up
```

### Feeding EVM transactions to the node

If the acala node is running with `--instant-sealing`, it won't start producing blocks without transactions coming in, and subquery indexer won't start either without any new blocks. So we need to feed some transactions to it.
This will run all 3 services together:
- a postgres database
- a subql node indexer that indexes data into postgres
- a subql query service that wraps the data with GraphQl interface, so we can query the data via GraphQl API

For example we can run any of the [evm examples](https://github.com/AcalaNetwork/bodhi.js/tree/master/examples).
Note that docker compose is for demo purpose, and in production we usually need to start each of the `{ postgres, indexer, query }` seperately, so it's more flexible and controllable.

```shell
cd ../../examples/waffle/dex
yarn build
yarn test
subql node should output something like this
```

If there are any transactions in the node, we should see the subquery indexer log something similar to this:

```shell
<BlockDispatcherService> INFO Enqueing blocks 3...12, total 10 blocks
<BlockDispatcherService> INFO fetch block [3,12], total 10 blocks
...
INFO Node started on port: 3000
INFO Enqueueing blocks 1102550...1102570, total 20 blocks
INFO Enqueueing blocks 1102570...1102590, total 20 blocks
```

## Query Data
We start indexing from block 1102550 for Acala as defined in the [config file](./project-acala.ts), since the first acala evm tx is at block 1102550.

Now we can explore the GraphQl data at [`http://localhost:3001/`](http://localhost:3001/) 🎉🎉

For example we can query:
### query the data
We now check if the whole stack works as expected by querying the data via subql query service at `http://localhost:3001`

```graphql
query {
Expand Down Expand Up @@ -163,59 +77,102 @@ query {
}
```

## For Production
### Connect with eth rpc
As the subql query is running, we can now start the eth rpc with `SUBQL_URL=http://localhost:3001` so that it can query the data from subql query service.

Previous examples are examples of the **local development setup**, which uses the [example configuration](./project.yaml) that is tailored to local development node.
For example, when we query `eth_getTransactionByHash` or `eth_getLogs`, the eth rpc will internally query the requested data from subql query. ([more details](https://evmdocs.acala.network/miscellaneous/faqs#when-do-i-need-to-provide-subquery-url-for-eth-rpc-adpater-or-evmrpcprovider))

For production deployment, there are a couple differences:
## Build and Run Locally with npm
For production it's recommended to use [docker setup](#run-with-docker), but if you don't want to use Docker, you can still run each service locally ([subquery docs](https://academy.subquery.network/run_publish/run.html#running-an-indexer-subql-node)).

#### services
In local setup we can run all of the services together with one single [docker compose](./docker-compose.yml). However, in prod we usually need to start each of the `{ node, postgres, indexer, query }` seperately with Docker or k8s.
install deps and build subql types and code locally
```
yarn
yarn build
```

#### image
In the local example, we use `subquerynetwork/subql-node-substrate:v3.10.1` as indexer image, which requires **local mounted project path**. For prod we should use [acala/evm-subql](https://hub.docker.com/r/acala/evm-subql/tags) instead, which already has all the required files encapsulated, so we don't need to mount local files anymore.
install subql cli globally
```shell
npm i -g @subql/[email protected] @subql/[email protected]
```

An example is [here](../docker-compose-example.yml#L27)
run a postgres db (in the second terminal). For production you can use other postgres providers such as AWS RDS, this is only for demo purpose.
```shell
docker run -it -p 5432:5432 -e POSTGRES_PASSWORD=postgres postgres:12-alpine
```

run a subql node indexer (in the third terminal)
```shell
export TZ=UTC
export DB_USER=postgres
export DB_PASS=postgres
export DB_DATABASE=postgres
export DB_HOST=localhost
export DB_PORT=5432

subql-node \
-f ./project-acala.yaml \
--network-endpoint wss://acala-rpc.aca-api.network \
--db-schema evm-acala \
--batch-size 20 \
--workers 2 \
--store-flush-interval 10 \
--log-level info \
--disable-historical \
--scale-batch-size \
--unsafe
```

Latest stable versions:
- `acala/eth-rpc-adapter:2.8.1`
- `acala/evm-subql:2.8.1`
- `subquerynetwork/subql-query:v2.10.0`
run a subql query service (in the fourth terminal)
```shell
export DB_USER=postgres
export DB_PASS=postgres
export DB_DATABASE=postgres
export DB_HOST=localhost
export DB_PORT=5432

#### config
One trick is that we don't have to start indexing from block 0, since Acala and Karura didn't enable EVM+ until a certain block. In particular we can use these two configs for production (change the `endpoint` value to your custom one if needed):
subql-query \
--name evm-acala \
--playground \
--indexer http://localhost:3000
```

now we should be able to [query the data](#query-the-data) via `http://localhost:3001`, and [connect with eth rpc](#connect-with-eth-rpc).

## Latest Stable Versions
- eth rpc: `acala/eth-rpc-adapter:2.9.4`
- subql node: `acala/evm-subql:2.9.4` (or `@subql/[email protected]`)
- subql query: `subquerynetwork/subql-query:v2.10.0` (or `@subql/[email protected]`)

- [Acala production](./project-acala-840000.yaml), by setting `-f=/app/project-acala-840000.yaml`
- [Karura production](./project-karura-1780000.yaml), by setting `-f=/app/project-karura-1780000.yaml`
[release page](https://github.com/AcalaNetwork/bodhi.js/releases)

## Upgrade Subquery
To upgrade subql, we usually **DO NOT** need to reindex from the beginning, which means we don't need to clear the db. We just need to upgrade the `acala/evm-subql` image version and restart the indexer, it should just continue indexing from the last indexed block. We also usually **DO NOT** need to touch subql query service.

It usually takes 1 to 3 days to index all of the data, depending on the node latency and performance.
If you are running subql node locally, just pull the latest code and run `yarn build`, then run `yarn index` again to restart the indexer.

### Upgrade Production Subquery
To upgrade the production subql, we usually need to do a full re-index. In order not to interrupt the currnetly running subql, we can run another indexer in parallel to the old one, and hot replace the old one once the full re-index is finished.
In rare cases, we might need to do a full re-index (we will explicitly state in the release note when this is needed). In order not to interrupt the currnetly running subql, we can run another indexer in parallel to the old one, and hot replace the old one once the full re-index is finished.

Below are the detailed steps:
1) start a new indexer service `acala/evm-subql` that uses a difference `--db-schema`, for example, `--db-schema=evm-acala`. It can share the same DB with the old indexer
1) start a new indexer service `acala/evm-subql` that uses a difference `--db-schema`, for example, `--db-schema=evm-acala-2`. It can share the same DB with the old indexer
2) wait until the new indexer finish indexing
3) update the config of graphql service `subquerynetwork/subql-query` to use the new indexer. In particular, change the `--name` command, such as `--name=evm-acala`, and `--indexer=<new indexer url>`
3) update the config of graphql service `subquerynetwork/subql-query` to use the new indexer. In particular, change the `--name` command to `--name=evm-acala-2`, and `--indexer=<new subql node url>`
4) delete the old indexer service, as well as the old db schema
5) upgrade is finished! No need to modify `eth-rpc-adapter`

Note: for `acala/evm-subql:v2.5.9` please add `--disable-historical` command. ([example](https://github.com/AcalaNetwork/bodhi.js/blob/d763bc588a4a90e4421d65ebfe1d95ba581c6d37/evm-subql/docker-compose.yml#L52))

## Dump and Restore Database
Sometimes it's useful to take a snapshot of the database, so that we can restore it when needed. We can also pass it along to others, so that they can quickly setup a copy of the same evm subql project, without needing to index from the beginning.

Below are CLI commands to do it, you can also use pgAdmin GUI to achieve the same thing.

### install postgres CLI
make sure you have `pg_dump` and `pg_restore` commands available.
make sure you have `pg_dump` and `pg_restore` commands available.

- for Mac: `brew install libpq`
- for other OS: `you are on your own`

### dump database
suppose we have an `evm-karura` schema in `postgres` db, and we want to dump it to a tar file `evm-karura.tgz`.
suppose we have an `evm-acala` schema in `postgres` db, and we want to dump it to a tar file `evm-acala.tgz`.
```
export PGPASSWORD=<password>
pg_dump \
Expand All @@ -224,13 +181,13 @@ pg_dump \
--dbname postgres \
--username postgres \
--format tar \
--file ./evm-karura.tgz \
--schema evm-karura \
--file ./evm-acala.tgz \
--schema evm-acala \
--verbose
```

### restore database
in previous step we dumped data from `evm-karura` schema in database `postgres`, so when restoring data, we need to first make sure a db called `postgres` exists, and it does **NOT** have `evm-karura` schema. Then we can restore the database with the following command
in previous step we dumped data from `evm-acala` schema in database `postgres`, so when restoring data, we need to first make sure a db called `postgres` exists, and it does **NOT** have `evm-acala` schema. Then we can restore the database with the following command
```
export PGPASSWORD=<password>
pg_restore \
Expand All @@ -239,20 +196,19 @@ pg_restore \
--dbname postgres \
--username postgres \
--verbose \
./evm-karura.tgz
./evm-acala.tgz
```

### (optional) rename schema
Since we dumped `evm-karura` schema, the restore process will create a new schema with the same name. If you want to use a different name, you can simply rename `evm-karura` schema to the desired name after the restore process.
### rename schema (optional)
Since we dumped `evm-acala` schema, the restore process will create a new schema with the same name. If you want to use a different name, you can simply rename `evm-acala` schema to the desired name after the restore process.

This can be done with pgAdmin by:
- right click the schema name
- select "properties"
- enter a new name and save

## More References

- [SubQuery official documentation](https://doc.subquery.network/)
- [About the unsafe flag](https://academy.subquery.network/run_publish/references.html#unsafe-node-service)
- [Acala EVM+ documentation](https://evmdocs.acala.network/network/network-setup/local-development-network)
- [Acala EVM+ documentation](https://evmdocs.acala.network/)
- [Subql Quick Migration Guide](https://hackmd.io/Z3ka28y4Tky6sHPsQVt2lw)
46 changes: 12 additions & 34 deletions packages/evm-subql/docker-compose.yml
View file
Original file line number Diff line number Diff line change
@@ -1,26 +1,6 @@
version: '3.9'

services:
mandala-node:
image: ghcr.io/acalanetwork/mandala-node:sha-16f147e
ports:
- 9944:9944
healthcheck:
test: "curl --fail -X POST -H \"Content-Type: application/json\" http://localhost:9944 -d \"{\"jsonrpc\": \"2.0\" }\" || exit 1"
interval: 2s
retries: 100
start_period: 3s
command:
- --dev
# - -lruntime=debug
# - -levm=debug
- --rpc-external
- --rpc-cors=all
- --rpc-methods=unsafe
- --pruning=archive
- --instant-sealing
- --tmp

postgres:
image: postgres:12-alpine
container_name: postgres
Expand All @@ -35,11 +15,9 @@ services:
retries: 5

subquery-node:
image: onfinality/subql-node:v5.2.9
image: acala/evm-subql:2.9.4 # built from onfinality/subql-node:v5.2.9
depends_on:
postgres:
condition: service_healthy
mandala-node:
postgres:
condition: service_healthy
ports:
- 3000:3000
Expand All @@ -50,17 +28,17 @@ services:
DB_DATABASE: postgres
DB_HOST: postgres
DB_PORT: 5432
volumes:
- ./:/app
command:
- ${SUB_COMMAND:-} # set SUB_COMMAND env variable to "test" to run tests
- -f=/app/project.yaml
- --network-endpoint=ws://mandala-node:9944
- --db-schema=acala_evm
- --log-level=debug
- --unsafe
- -f=./project-acala.yaml
- --network-endpoint=wss://acala-rpc.aca-api.network # acala node url
- --db-schema=evm-acala
- --batch-size=20
- --workers=2
- --store-flush-interval=10
- --log-level=info
- --disable-historical
- --batch-size=30
- --scale-batch-size
- --unsafe
healthcheck:
test: ["CMD", "curl", "-f", "http://subquery-node:3000/ready"]
interval: 3s
Expand All @@ -85,7 +63,7 @@ services:
DB_PORT: 5432
PORT: 3001
command:
- --name=acala_evm
- --name=evm-acala
- --playground
- --indexer=http://subquery-node:3000

Expand Down
4 changes: 1 addition & 3 deletions packages/evm-subql/package.json
View file
Original file line number Diff line number Diff line change
Expand Up @@ -9,9 +9,7 @@
"clean": "rm -rf dist",
"codegen": "./codegen.sh",
"prepack": "yarn build",
"build": "yarn codegen; ncc build src/index.ts -o dist/ -t --target es2017; ncc build src/chain-types.ts -o dist/chain-types -t --target es2017",
"index": "subql-node -f ./project.yaml --batch-size 30 --db-schema acala_evm --log-level debug --port 3000 --unsafe --disable-historical",
"query": "PORT=3001 subql-query --name acala_evm --playground --indexer=http://localhost:3000 --debug"
"build": "yarn codegen; ncc build src/index.ts -o dist/ -t --target es2017; ncc build src/chain-types.ts -o dist/chain-types -t --target es2017"
},
"dependencies": {
"@acala-network/eth-providers": "workspace:*"
Expand Down
2 changes: 1 addition & 1 deletion packages/evm-subql/project-acala.yaml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ network:
file: ./dist/chain-types/index.js
dataSources:
- kind: substrate/Runtime
startBlock: 1
startBlock: 1102550
mapping:
file: ./dist/index.js
handlers:
Expand Down
Loading

0 comments on commit 5d78934

Please sign in to comment.