Merge pull request #691 from blockscout/lymarenkolev/bens/gnosis-graph

add genome subgraph + add subgraph creation tool

blockscout-ens/.gitignore

@@ -1,2 +1,3 @@
 data
-node_modules
+node_modules
+.env

blockscout-ens/README.md

@@ -0,0 +1,19 @@
+# Blockscout ENS service
+
+This project provides indexed data of domain name service for blockscout instances.
+
+Here is brief overview of the project stucture:
+
+![bens-structure](images/bens.drawio.svg)
+
+Service is **multi-chain**, meaning that only one instance of `graph-node`, `postgres` and `bens-server` is required.
+
+## Contribute
+
+If you want to add your name service procol to blockscout you should:
+
+1. Clone this `blockscout-rs` repo to add new protocol.
+1. Write subraph code: read [subgraph writer guide](./graph-node/subgraph-writer/README.md#howto-create-subgraph-for-your-domain-name-protocol)
+1. Start graph-node: read [graph-node guide](./graph-node/README.md#start-locally-using-docker-compose)
+1. Deploy subgraph to graph-node: read [how to deploy subgraphs guide](./graph-node/subgraphs/README.md#deploy-subgraph-to-graph-node)
+1. Start `bens-server` connected to common database: read [how to start bens guide](./bens-server/README.md#to-start-locally)

blockscout-ens/bens-server/README.md

@@ -0,0 +1,11 @@
+# Bens-server
+
+## To start locally
+
+1. Compile and run:
+
+    ```bash
+    export BENS__DATABASE__CONNECT__URL="<database-url>"
+    export BENS__BLOCKSCOUT__NETWORKS__<chain_id>__URL=<blockscout_url>
+    cargo run --bin bens-server
+    ```

blockscout-ens/graph-node/README.md

@@ -4,22 +4,22 @@
 
 One can submit subgraph to `graph-node` -- actual code how to handle new events of contracts
 
-## Start locally
+## Start locally using docker-compose
 
-+ Edit `docker-compose.yml` and change `ethereum` ENV variable of `graph-node` services to add your own network and RPC url.
+1. Edit `docker-compose.yml` and change `ethereum` ENV variable of `graph-node` services to add your own network and RPC url.
 
-+ Start your own graph-node with docker-compose:
+1. Start your own graph-node with docker-compose:
 
-```bash
-docker-compose up -d
-```
+    ```bash
+    docker-compose up -d
+    ```
 
-+ Load small version of ens-rainbow
+1. Load small version of ens-rainbow
 
-```bash
-./rainbow.small.sh
-```
+    ```bash
+    ./rainbow.small.sh
+    ```
 
-+ Or use [full ens-rainbow](https://github.com/graphprotocol/ens-rainbow/) dump
+    Or use [full ens-rainbow](https://github.com/graphprotocol/ens-rainbow/) dump if you want full domain name resolving
 
-+ Read [subgraphs/README.md](./subgraphs/README.md) to build and deploy subgraph to graph-node
+1. Read [subgraphs/README.md](./subgraphs/README.md) to build and deploy subgraph to graph-node

blockscout-ens/graph-node/docker-compose.yml

@@ -20,7 +20,7 @@ services:
       postgres_pass: let-me-in
       postgres_db: graph-node
       ipfs: 'ipfs:5001'
-      ethereum: 'mainnet:https://api.zmok.io/mainnet/oaen6dy8ff6hju9k'
+      ethereum: 'mainnet:https://api.zmok.io/mainnet/oaen6dy8ff6hju9k gnosis-chiado:https://rpc.eu-central-2.gateway.fm/v3/gnosis/archival/chiado'
       GRAPH_LOG: info
     volumes:
       - ./gn-config.toml:/root/gn-config.toml
@@ -33,7 +33,7 @@ services:
       - ./data/ipfs:/data/ipfs
   postgres:
     container_name: gn-postgres-dev
-    image: postgres:14
+    image: postgres:16.1
     ports:
       - '5432:5432'
     command:

blockscout-ens/graph-node/rainbow.small.sh

@@ -28,7 +28,7 @@ CREATE TABLE public.ens_names (
 -- Data for ens_names
 --
 
-COPY public.ens_names (hash, name) FROM stdin ON CONFLICT DO NOTHING/UPDATE;
+COPY public.ens_names (hash, name) FROM stdin;
 0x6817c00f03de8b5bd58d2016b59d251c13056b989171c5852949903bc043bc27	get
 0x7a9332fa0d910db7dd4d354d0e6667ab1ebcb9fdcc6a0025433332ec7fa15e2e	blogger
 0xc0da3fe9cba3aac3696e9a7fb1f0aadd23e31f6deeca7d3d74c8d9d0e7621cd8	microsoft

blockscout-ens/graph-node/subgraph-writer/.gitignore

@@ -0,0 +1,3 @@
+.venv
+.env
+__pycache__

blockscout-ens/graph-node/subgraph-writer/README.md

@@ -0,0 +1,50 @@
+
+# Howto create subgraph for your domain name protocol
+
+The first thing to note is that the closer your protocol is to **ENS**, the easier it will be to create a blockscout-compatible subgraph.
+We take initial structure from [ENS subgraph](https://github.com/ensdomains/ens-subgraph).
+You can take a look at that subgraph and understand structure of our project more precisely.
+
+1. Install [just](https://github.com/casey/just). `Just` is like cmake but better.
+
+1. Install python3 and install deps:
+
+    ```bash
+    just init
+    ```
+
+1. Now you have to create file inside `protocols` directory decribing your procol. Use `example.protocol.yaml` as template.
+
+1. You can try to generate protocol desription file using `protocol-extractor`. This script will try to extract verified contracts from etherscan and determine their affiliation with the protocol:
+
+    ```bash
+    just try-build-protocol <protocol-name> <etherscan-endpoint-with-api-key> <addresses-of-contracts-comma-separated>
+    ```
+
+    This command will create `protocols/<name>.yaml` file with decription of contracts. You still need to add `project_name` and other metadata field. Also change generated fields it if necessary.
+
+1. Generate subgraph project from template:
+
+    ```bash
+    just template-it-from-protocol protocols/<name>.yaml ../subgraphs
+    ```
+
+    This command will create project inside `../subgraphs/<project_name>`
+
+1. Move to recently created directory and run
+
+    ```bash
+    just init && just codegen
+    ```
+
+    In case of any error, adjust typescript code of subgraph. Also make sure subgraph handles events properly.
+
+1. Write your mappings: read [official subgraph guide](https://thegraph.com/docs/en/developing/creating-a-subgraph/#writing-mappings). You have to handle events of your protocol properly in order to index all blockchain data. You can use default mapping from generated template, however make sure that code is written correctly.
+
+1. Now build subgraph code
+
+    ```bash
+    just build
+    ```
+
+1. Now you should run your subgraph by submitting it to graph-node: read [deploy subgraphs to graph-node](../subgraphs/README.md#deploy-subgraph-to-graph-node)

blockscout-ens/graph-node/subgraph-writer/justfile

@@ -0,0 +1,29 @@
+init:
+    python -m pip install -r requirements.txt
+
+try-build-protocol name etherscan_endpoint addresses:
+    python protocol-extractor/get_abi_from_etherscan.py \
+        --endpoint {{etherscan_endpoint}} \
+        --output protocol-extractor/output-abis.json \
+        --addresses {{addresses}}
+    python protocol-extractor/extractor.py \
+        --config protocol-extractor/contract_events.json \
+        --input-abis protocol-extractor/output-abis.json \
+        --output protocols/{{name}}.yaml
+
+template-it-from-protocol protocol output_dir:
+    # todo: -o ../subgraphs
+    cookiecutter --config-file {{protocol}} --output-dir {{output_dir}} --no-input ./templater
+
+
+test-ens MAINNET_ETHERSCAN_API_KEY:
+    just try-build-protocol \
+        test-ens \
+        https://api.etherscan.io/api\?apikey\={{MAINNET_ETHERSCAN_API_KEY}} \
+        0x00000000000C2E074eC69A0dFb2997BA6C7d2e1e,0x253553366Da8546fC250F225fe3d25d0C782303b,0x57f1887a8BF19b14fC0dF6Fd9B2acc9Af147eA85,0x231b0Ee14048e9dCcD1d247744d114a4EB5E8E63
+
+test-genome GNOSIS_ETHERSCAN_API_KEY:
+    just try-build-protocol \
+        test-genome \
+        https://api.gnosisscan.io/api\?apikey\={{GNOSIS_ETHERSCAN_API_KEY}} \
+        0xFFC305697bb27483268106be6F22B0555c192804,0x3D3dc123c60c2172c63B1E4D4C026Eb77fb57Bd9,0x4211B1Dcf3adbEbAB6013A907465d2B90e90eb6d,0xFa8093b53AE47DfD24E563F124fA7206089c502a,0x57410305546fe40062144BF182ded612Bb8F13B8,0x3802AB1E503ab373D370d71386DFEe33851e32A9,0xc9e1a0aDdE5168313D5D683c75813100B798374a,0x720DC4afE863fe467cDC00a768617721b912FbAe,0x3250e011E77c462f69D73275A2Da4fbba55181d5,0xC742CB159ccEc58D99CE8F4acb51271288783a82,0xc6DB15d9cDb0402c10A6091855d3c90bb3e4FD1A,0xB21A2AFfC24683b464505160375E0c7D4F8af397,0x9C42F51bb8f71EC8b9A8E22D675226D3aC312208,0x6FdcF80a80dB740f88021232AD58341232Ba61b2,0x51e3e07667A0bCb2f4807640dCB61fd38d7829AF,0xd8c34a379BD95c3EfFF2CA23Dfa27D83962dc44a

blockscout-ens/graph-node/subgraph-writer/protocol-extractor/.gitignore

@@ -0,0 +1 @@
+output-*.json

blockscout-ens/graph-node/subgraph-writer/protocol-extractor/contract_events.json

@@ -0,0 +1,177 @@
+{
+  "registry": {
+    "default_name": "Registry",
+    "events": [
+      {
+        "name": "NewOwner",
+        "inputs": [
+          {
+            "indexed": true,
+            "name": "node",
+            "type": "bytes32"
+          },
+          {
+            "indexed": true,
+            "name": "label",
+            "type": "bytes32"
+          },
+          {
+            "indexed": false,
+            "name": "owner",
+            "type": "address"
+          }
+        ]
+      },
+      {
+        "name": "NewResolver",
+        "inputs": [
+          {
+            "indexed": true,
+            "name": "node",
+            "type": "bytes32"
+          },
+          {
+            "indexed": false,
+            "name": "resolver",
+            "type": "address"
+          }
+        ]
+      },
+      {
+        "name": "NewTTL",
+        "inputs": [
+          {
+            "indexed": true,
+            "name": "node",
+            "type": "bytes32"
+          },
+          {
+            "indexed": false,
+            "name": "ttl",
+            "type": "uint64"
+          }
+        ]
+      },
+      {
+        "name": "Transfer",
+        "inputs": [
+          {
+            "indexed": true,
+            "name": "node",
+            "type": "bytes32"
+          },
+          {
+            "indexed": false,
+            "name": "owner",
+            "type": "address"
+          }
+        ]
+      }
+    ]
+  },
+  "controller": {
+    "default_name": "RegistrarController",
+    "events": [
+      {
+        "name": "NameRegistered",
+        "inputs": [
+          {
+            "indexed": false,
+            "name": "name",
+            "type": "string"
+          },
+          {
+            "indexed": true,
+            "name": "label",
+            "type": "bytes32"
+          },
+          {
+            "indexed": true,
+            "name": "owner",
+            "type": "address"
+          }
+        ]
+      },
+      {
+        "name": "NameRenewed",
+        "inputs": [
+          {
+            "indexed": false,
+            "name": "name",
+            "type": "string"
+          },
+          {
+            "indexed": true,
+            "name": "label",
+            "type": "bytes32"
+          },
+          {
+            "indexed": false,
+            "name": "expires",
+            "type": "uint256"
+          }
+        ]
+      }
+    ]
+  },
+  "resolver": {
+    "default_name": "PublicResolver",
+    "events": [
+      {
+        "inputs": [
+          {
+            "indexed": true,
+            "name": "node",
+            "type": "bytes32"
+          },
+          {
+            "indexed": false,
+            "name": "a",
+            "type": "address"
+          }
+        ],
+        "name": "AddrChanged"
+      }
+    ]
+  },
+  "base": {
+    "default_name": "BaseRegistrar",
+    "events": [
+      {
+        "inputs": [
+          {
+            "indexed": true,
+            "name": "id",
+            "type": "uint256"
+          },
+          {
+            "indexed": true,
+            "name": "owner",
+            "type": "address"
+          },
+          {
+            "indexed": false,
+            "name": "expires",
+            "type": "uint256"
+          }
+        ],
+        "name": "NameRegistered"
+      },
+      {
+        "inputs": [
+          {
+            "indexed": true,
+            "name": "id",
+            "type": "uint256"
+          },
+          {
+            "indexed": false,
+            "name": "expires",
+            "type": "uint256"
+          }
+        ],
+        "name": "NameRenewed"
+      }
+    ]
+  }
+}