Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Proposal: Arb-subgraph refactoring #585

Open
wants to merge 7 commits into
base: main
Choose a base branch
from
Open

Proposal: Arb-subgraph refactoring #585

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
3054e72
feat: refactor arb-subgraph forta alert, updated packages, added pret…
Tarens2 Jul 5, 2024
fa7d8a6
feat: fixed warnings, removed old yarn version
Tarens2 Jul 5, 2024
feac5c6
Merge branch 'main' into feature/si-1316-refactor-subgraph-forta-alert
Tarens2 Jul 16, 2024
ec01887
added eslint config
Tarens2 Jul 16, 2024
c20cbc4
Merge branch 'main' of github.com:lidofinance/alerting-forta into fea…
DiRaiks Jul 25, 2024
83902d1
feat: refactoring subgraph alert to fastify package
DiRaiks Jul 26, 2024
2b9b02c
fix: remove unnecessary files
DiRaiks Jul 26, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions arb-subgraph/.eslintignore
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
generated
proto
14 changes: 14 additions & 0 deletions arb-subgraph/.eslintrc.json
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
{
"parser": "@typescript-eslint/parser",
"extends": ["eslint:recommended", "plugin:@typescript-eslint/recommended", "prettier"],
"plugins": ["@typescript-eslint", "prettier"],
"env": {
"node": true,
"es6": true
},
"rules": {
"prettier/prettier": "error",
"curly": "error",
"semi": "off"
}
}
3 changes: 3 additions & 0 deletions arb-subgraph/.gitignore
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,3 +6,6 @@ forta.config.json
*.log
version.json
.DS_Store

# Env
.env
2 changes: 2 additions & 0 deletions arb-subgraph/.prettierignore
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
generated
proto
7 changes: 7 additions & 0 deletions arb-subgraph/.prettierrc.json
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
{
"semi": false,
"trailingComma": "all",
"singleQuote": true,
"printWidth": 120,
"tabWidth": 2
}
807 changes: 0 additions & 807 deletions arb-subgraph/.yarn/releases/yarn-3.3.0.cjs
View file
Open in desktop

This file was deleted.

3 changes: 0 additions & 3 deletions arb-subgraph/.yarnrc.yml
View file
Open in desktop

This file was deleted.

15 changes: 5 additions & 10 deletions arb-subgraph/Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,14 +1,13 @@
# Build stage: compile Typescript to Javascript
FROM node:16.17.1-alpine3.16 AS base
FROM node:20.9.0-alpine3.18 AS base
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would suggest using the latest node.

FROM node:20.15.1-alpine3.20 AS base

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This was left as is. The refactoring only affected the code of the bot itself. Thanks - I'll fix it!


FROM base as builder
WORKDIR /app

COPY package*.json .yarnrc.yml yarn.lock ./
COPY .yarn/releases ./.yarn/releases/
RUN yarn install --immutable
WORKDIR /app

COPY . .
RUN yarn install --immutable && yarn run build
# Build app
RUN yarn run build

# Final stage: copy compiled Javascript from previous stage and install production dependencies
Expand All @@ -17,13 +16,9 @@ LABEL "network.forta.settings.agent-logs.enable"="true"
ENV NODE_ENV=production
WORKDIR /app

Copy link
Contributor

@sergeyWh1te sergeyWh1te Jul 30, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You have to provide the default values in DockerFile.
Forta knows nothing about your custom envs

RUN apk add --no-cache tini=0.19.0-r0

COPY package*.json .yarnrc.yml yarn.lock ./
COPY .yarn/releases ./.yarn/releases/
COPY package*.json yarn.lock ./
COPY --from=builder /app/node_modules ./node_modules
COPY --from=builder /app/dist ./src
COPY version.json ./

ENTRYPOINT ["/sbin/tini", "--"]
CMD ["yarn", "run", "start:prod"]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure that it would work. Check pls.
If it does not work - here is a working combination:

CMD ["yarn", "run", "start:docker:prod"]

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ill check it

42 changes: 42 additions & 0 deletions arb-subgraph/Makefile
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,42 @@
# Makefile

.PHONY: gen_proto_prod
gen_proto_prod:
# generate js codes via grpc-tools
yarn grpc_tools_node_protoc \
--js_out=import_style=commonjs,binary:./dist/generated/proto \
--grpc_out=grpc_js:./dist/generated/proto \
--plugin=protoc-gen-grpc=./node_modules/.bin/grpc_tools_node_protoc_plugin \
-I ./src/brief/proto \
./src/brief/proto/*.proto

.PHONY: gen_js
gen_js:
# generate js codes via grpc-tools
yarn grpc_tools_node_protoc \
--js_out=import_style=commonjs,binary:./src/generated/proto \
--grpc_out=grpc_js:./src/generated/proto \
--plugin=protoc-gen-grpc=./node_modules/.bin/grpc_tools_node_protoc_plugin \
--proto_path=./src/brief/proto \
./src/brief/proto/*.proto

.PHONY: gen_ts
gen_ts:
# generate d.ts codes
yarn grpc_tools_node_protoc \
--plugin=protoc-gen-ts=./node_modules/.bin/protoc-gen-ts \
--ts_out=grpc_js:./src/generated/proto \
--proto_path=./src/brief/proto \
./src/brief/proto/*.proto

tools:
cd tools && go mod tidy && go mod vendor && go mod verify && go generate -tags tools
.PHONY: tools

.PHONY: check_alerts_syntax
check_alerts_syntax:
./bin/promtool check rules ./alerts.yml

.PHONY: test_alerts
test_alerts:
bin/promtool test rules ./alerts_tests.yml
10 changes: 5 additions & 5 deletions arb-subgraph/jest.config.js
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
/** @type {import('ts-jest').JestConfigWithTsJest} */
/** @type {import("ts-jest").JestConfigWithTsJest} */
module.exports = {
preset: "ts-jest",
testEnvironment: "node",
testRegex: ".*\\.spec\\.ts$",
};
preset: 'ts-jest',
testEnvironment: 'node',
testPathIgnorePatterns: ['dist'],
}
75 changes: 52 additions & 23 deletions arb-subgraph/package.json
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,7 +1,8 @@
{
"name": "lido-arbitrum-bot",
"name": "arb-subgraph",
"version": "0.0.1",
"description": "Forta Bot for Lido Apps on Arbitrum network",
"description": "Forta Bot for Lido The Graph billing account",
"license": "MIT",
"chainIds": [
42161
],
Expand All @@ -11,34 +12,62 @@
"target": 5
}
},
"husky": {
"hooks": {
"pre-commit": "yarn run lint"
}
},
"scripts": {
"update-version": "node scripts/write-version.js",
"build": "tsc --resolveJsonModule",
"start": "yarn run start:dev",
"start:dev": "nodemon --watch src --watch forta.config.json -e js,ts,json --exec \"yarn run build && forta-agent run\"",
"start:prod": "forta-agent run --prod",
"tx": "yarn run build && forta-agent run --tx",
"block": "yarn run build && forta-agent run --block",
"range": "yarn run build && forta-agent run --range",
"file": "yarn run build && forta-agent run --file",
"publish": "yarn run update-version && forta-agent publish",
"update-version": "node ../utils/write-version.js",
"build": "tsc && yarn run copy-version && mkdir dist/generated/proto && make gen_proto_prod",
"copy-version": "cp version.json dist",
"start": "ts-node src/main.ts",
"start:dev": "nodemon --watch src --watch forta.config.json -e js,ts,json --exec \"yarn run build && yarn run copy-version && forta-agent run\"",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Useless command. This commit removes forta-agent dependency

"start:prod": "node dist/main.js",
"push": "yarn run update-version && forta-agent push",
"disable": "forta-agent disable",
"enable": "forta-agent enable",
"keyfile": "forta-agent keyfile",
"test": "jest --passWithNoTests"
"test": "jest",
"generate-types": "typechain --target=ethers-v5 --out-dir=./src/generated/typechain ./src/brief/abi/*",
"generate-proto": "make gen_ts && make gen_js",
"eslint:lint": "eslint ./src",
"eslint:format": "eslint ./src --fix",
"prettier:check": "prettier --check ./src",
"prettier:format": "prettier --write ./src",
"lint": "yarn run prettier:check && yarn run eslint:lint",
"format": "yarn run eslint:format && yarn run prettier:format",
"postinstall": "yarn generate-types && yarn generate-proto"
},
"dependencies": {
"bignumber.js": "^9.0.1",
"forta-agent": "^0.1.48"
"@fastify/env": "^4.4.0",
"bignumber.js": "^9.1.2",
"fastify": "^4.28.1",
"fastify-plugin": "^4.5.1",
"forta-agent": "^0.1.48",
"prom-client": "^15.1.3",
"winston": "^3.13.1"
},
"devDependencies": {
"@types/jest": "^29.1.2",
"@ethersproject/abi": "^5.0.0",
"@ethersproject/providers": "^5.0.0",
"@tsconfig/node20": "^20.1.2",
"@typechain/ethers-v5": "^11.1.2",
"@types/jest": "^29.5.12",
"@types/nodemon": "^1.19.0",
"jest": "^29.2.0",
"nodemon": "^2.0.8",
"@typescript-eslint/eslint-plugin": "^6.12.0",
"@typescript-eslint/parser": "^6.12.0",
"eslint": "^8.54.0",
"eslint-config-prettier": "^9.0.0",
"eslint-plugin-jest": "^27.6.0",
"eslint-plugin-prettier": "^5.0.1",
"ethers": "^5.1.3",
"grpc-tools": "^1.12.4",
"grpc_tools_node_protoc_ts": "^5.3.3",
"husky": "^8.0.3",
"jest": "^29.7.0",
"nodemon": "^3.0.1",
"prettier": "^3.1.0",
"ts-jest": "^29.0.3",
"typescript": "^4.3.4"
"typechain": "^8.3.2",
"typescript": "^5.3.2"
},
"packageManager": "yarn@3.3.0"
"packageManager": "yarn@1.22.21"
}
Copy link
Contributor

@sergeyWh1te sergeyWh1te Jul 30, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would suggest upgrading all deps in package.json

6 changes: 6 additions & 0 deletions arb-subgraph/sample.env
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@

# The URL of the Arbitrum RPC endpoint
ARBITRUM_RPC_URL=https://arbitrum.drpc.com
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Found that https://arbitrum-one.publicnode.com works better. Especially for reading some data in the past.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a sample for envs. I doubt that these URLs will be used in any work of the bot


# Port for the HTTP server
HTTP_PORT=3000
33 changes: 12 additions & 21 deletions arb-subgraph/scripts/write-version.js
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,23 +1,14 @@
const childProcess = require("child_process");
const fs = require("fs");
const childProcess = require('child_process')
const fs = require('fs')

const commitHash = childProcess
.execSync("git rev-parse HEAD")
.toString("utf-8")
.trim();
const commitMsg = childProcess
.execSync("git log -1 --pretty=%B")
.toString("utf-8")
.trim();
const gitStatusOutput = childProcess
.execSync("git status --porcelain")
.toString("utf-8")
.trim();
const commitHash = childProcess.execSync('git rev-parse HEAD').toString('utf-8').trim()
const commitMsg = childProcess.execSync('git log -1 --pretty=%B').toString('utf-8').trim()
const gitStatusOutput = childProcess.execSync('git status --porcelain').toString('utf-8').trim()

const commitHashShort = commitHash.substr(0, 7);
const commitMsgShort = commitMsg.split("\n")[0];
const isWdClean = gitStatusOutput === "";
const commitHashSuffux = isWdClean ? "" : " [dirty]";
const commitHashShort = commitHash.substr(0, 7)
const commitMsgShort = commitMsg.split('\n')[0]
const isWdClean = gitStatusOutput === ''
const commitHashSuffux = isWdClean ? '' : ' [dirty]'

const version = {
desc: `${commitHashShort}${commitHashSuffux} (${commitMsgShort})`,
Expand All @@ -26,8 +17,8 @@ const version = {
commitMsg,
commitMsgShort,
isWdClean,
};
}

console.log("Writing to version.json:", JSON.stringify(version, null, " "));
console.log('Writing to version.json:', JSON.stringify(version, null, ' '))

fs.writeFileSync("version.json", JSON.stringify(version));
fs.writeFileSync('version.json', JSON.stringify(version))
Loading
Loading