Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add websocket entrypoint for app #26

Merged
merged 32 commits into from
May 28, 2024
Merged
Show file tree
Hide file tree
Changes from 14 commits
Commits
Show all changes
32 commits
Select commit Hold shift + click to select a range
9e658d1
Add websocket entrypoint for app
Sotatek-PhucTa Apr 23, 2024
55c1f23
Add subscribe for event execute
Sotatek-JohnnyNguyen Apr 24, 2024
8ecfd0f
Integrate with communicate service for furthur handling
Sotatek-JohnnyNguyen Apr 24, 2024
48b4744
Add test for method not allowed
Sotatek-JohnnyNguyen Apr 24, 2024
f7a0ed8
feat: implement swith method when emit ws
Sotatek-JohnnyNguyen Apr 25, 2024
cf364c0
chore: define response for event ws
Sotatek-JohnnyNguyen Apr 26, 2024
4e55ad3
chore: update unit test for metho net_version
Sotatek-JohnnyNguyen May 2, 2024
18c2782
feat: change socket.io to websocket
Sotatek-JohnnyNguyen May 3, 2024
c643b49
feat: change unit test from socketio to pure socket
Sotatek-JohnnyNguyen May 4, 2024
895947c
feat: refactor code
Sotatek-William May 9, 2024
0b70808
feat: refactor code
Sotatek-William May 9, 2024
2a401ca
remove wsPort, add .env.example file and comment unit test
Sotatek-William May 10, 2024
de5a966
add eth_subscribe and eth_unsubscribe to allowedMethods array
Sotatek-William May 10, 2024
8b341d5
optimize code logic
Sotatek-William May 10, 2024
9e89e83
add step Set configuration variables to README file
Sotatek-William May 10, 2024
a4dbb2d
fix: check socket connection
Sotatek-William May 10, 2024
f5390b5
check rate limit if request is sendrawtransaction
Sotatek-William May 13, 2024
9687449
check rate limit if request is sendrawtransaction
Sotatek-William May 13, 2024
cff24e1
add-unit-test
Sotatek-William May 14, 2024
55f6bff
add-unit-test
Sotatek-William May 14, 2024
402e71e
improve unit test
Sotatek-William May 15, 2024
d21fe70
add reconnect method
Sotatek-William May 15, 2024
db29b85
add reconnect method
Sotatek-William May 15, 2024
85178a8
update README
Sotatek-William May 16, 2024
f829080
fix README and keep the connection even if error from node's websocke…
Sotatek-William May 16, 2024
15482a5
fix: issues reported by BOT
Sotatek-William May 17, 2024
62ff16c
update: communicate test file and README
Sotatek-JohnnyNguyen May 21, 2024
ca56fda
update: communicate test file and README
Sotatek-JohnnyNguyen May 21, 2024
d00874b
update: .env.example file
Sotatek-JohnnyNguyen May 21, 2024
b72b24f
update: communicate test file
Sotatek-JohnnyNguyen May 21, 2024
b789678
Apply formatter
ironbeer May 21, 2024
bf0e2f4
Merge pull request #27 from oasysgames/features/add-support-web-socke…
ironbeer May 21, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions .env.example
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
VERSE_MASTER_NODE_URL=
VERSE_URL=
VERSE_READ_NODE_URL=http://localhost:8545
BLOCK_NUMBER_CACHE_EXPIRE_SEC=
DATASTORE=
NODE_SOCKET=ws://[::]:8546

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Sotatek-JohnnyNguyen
.env.example file has been added, but how is it intended to be used?
It appears that dotenv is not included in the package.json.
Will you use the source .env command?

And it would be helpful if you could also update the README.md file to reflect the current situation.

Although it is unrelated to this change, I believe that $ npm build in the README.md file should be corrected to $ npm run build.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

First: Nestjs provide configModule that exposes a configService which loads the appropriate .env file, so we dont need dotenv for now
Second: the .env.example is like an example to help other devs know which configuration variables they have to add to .env file in order to run the service

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I will update README.me file now. Also yes both commands $ npm build and $ npm run build are doing the same action

Copy link

@georgefuru georgefuru May 10, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Sotatek-JohnnyNguyen
Okay. Thanks for replying.

First: Nestjs provide configModule that exposes a configService which loads the appropriate .env file, so we dont need dotenv for now

Good! Thanks.

Second: the .env.example is like an example to help other devs know which configuration variables they have to add to .env file in order to run the service

Of course, I know that. My way of asking the question might not have been good.

By the way, what do you think about including the environment variable PORT=, which is currently written to be exported in the README.md, in the .env.example file?
If there are no issues, I feel it would be good to add it.

Copy link
Contributor Author

@Sotatek-JohnnyNguyen Sotatek-JohnnyNguyen May 10, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sure

By the way, what do you think about including the environment variable PORT=, which is currently written to be exported in the README.md, in the .env.example file?
If there are no issues, I feel it would be good to add it.

3,084 changes: 1,789 additions & 1,295 deletions package-lock.json

Large diffs are not rendered by default.

13 changes: 11 additions & 2 deletions package.json
Original file line number Diff line number Diff line change
Expand Up @@ -14,8 +14,10 @@
"start:dev": "nest start --watch",
"start:debug": "nest start --debug --watch",
"start:prod": "node dist/main",
"console:dev": "ts-node -r tsconfig-paths/register src/console.ts",
"console": "node dist/console.js",
"lint": "eslint \"{src,apps,libs,test}/**/*.ts\" --fix",
"test": "jest",
"test": "jest --detectOpenHandles",
"test:watch": "jest --watch",
"test:cov": "jest --coverage",
"test:debug": "node --inspect-brk -r tsconfig-paths/register -r ts-node/register node_modules/.bin/jest --runInBand",
Expand All @@ -27,14 +29,20 @@
"@nestjs/common": "^9.0.0",
"@nestjs/config": "^2.2.0",
"@nestjs/core": "^9.0.0",
"@nestjs/event-emitter": "^2.0.4",
"@nestjs/platform-express": "^9.0.0",
"@nestjs/platform-socket.io": "^9.0.0",
"@nestjs/platform-ws": "^9.0.0",
"@nestjs/websockets": "^9.0.0",
"body-parser": "^1.20.2",
"ethers": "^5.7.1",
"ioredis": "^5.3.1",
"nestjs-real-ip": "^2.2.0",
"reflect-metadata": "^0.1.13",
"rimraf": "^3.0.2",
"rxjs": "^7.2.0"
"rxjs": "^7.2.0",
"socket.io-client": "^4.7.5",
"ws": "^8.17.0"
},
"devDependencies": {
"@nestjs/cli": "^9.0.0",
Expand All @@ -44,6 +52,7 @@
"@types/jest": "28.1.8",
"@types/node": "^16.0.0",
"@types/supertest": "^2.0.11",
"@types/ws": "^8.5.10",
"@typescript-eslint/eslint-plugin": "^5.0.0",
"@typescript-eslint/parser": "^5.0.0",
"eslint": "^8.0.1",
Expand Down
4 changes: 4 additions & 0 deletions src/app.module.ts
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,8 @@ import {
} from './services';
import { DatastoreService } from './repositories';
import configuration from './config/configuration';
import { CommunicateGateway } from './communicates/communicate.gateway';
import { WebSocketService } from './services/webSocket.sevice';

@Module({
imports: [
Expand All @@ -23,6 +25,8 @@ import configuration from './config/configuration';
],
controllers: [ProxyController],
providers: [
WebSocketService,
CommunicateGateway,
VerseService,
TransactionService,
ProxyService,
Expand Down
103 changes: 103 additions & 0 deletions src/communicates/__tests__/communicate.gateway.spec.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,103 @@
import { CacheModule, INestApplication, Logger } from '@nestjs/common';
import { CommunicateGateway } from '../communicate.gateway';
import { ConfigModule } from '@nestjs/config';
import { Test, TestingModule } from '@nestjs/testing';
import configuration from '../../config/configuration';
import { TypeCheckService } from 'src/services';
import { HttpModule } from '@nestjs/axios';
import * as WebSocket from 'ws';
import { WebSocketService } from 'src/services/webSocket.sevice';

async function createNestApp(...gateways: any): Promise<INestApplication> {

Check warning on line 11 in src/communicates/__tests__/communicate.gateway.spec.ts

View workflow job for this annotation

GitHub Actions / check code

'createNestApp' is defined but never used
const testingModule = await Test.createTestingModule({
imports: [
ConfigModule.forRoot({ load: [configuration] }),
HttpModule,
CacheModule.register(),
],
providers: gateways,
}).compile();
testingModule.useLogger(new Logger());
return testingModule.createNestApplication();
}

describe('Communicate gateway', () => {
let app: INestApplication;

Check warning on line 25 in src/communicates/__tests__/communicate.gateway.spec.ts

View workflow job for this annotation

GitHub Actions / check code

'app' is assigned a value but never used
let client: WebSocket;

Check warning on line 26 in src/communicates/__tests__/communicate.gateway.spec.ts

View workflow job for this annotation

GitHub Actions / check code

'client' is defined but never used
let moduleRef: TestingModule;
let webSocketService: WebSocketService;

Check warning on line 28 in src/communicates/__tests__/communicate.gateway.spec.ts

View workflow job for this annotation

GitHub Actions / check code

'webSocketService' is assigned a value but never used
beforeAll(async () => {
moduleRef = await Test.createTestingModule({
imports: [
ConfigModule.forRoot({
load: [configuration],
}),
],
controllers: [],
providers: [CommunicateGateway, TypeCheckService, WebSocketService],
}).compile();

app = moduleRef.createNestApplication();
webSocketService = moduleRef.get<WebSocketService>(WebSocketService);
// client = new WebSocket('http://localhost:3000');
});

// afterAll(async () => {
// if (client.readyState === client.OPEN) {
// client.removeAllListeners();
// client.close();
// }
// await app.close();
// });

it(`Should emit "pong" on "ping"`, (done) => {
done();
});

// it(`Should emit "pong" on "ping"`, (done) => {
// client.on('open', () => {
// client.send('ping');
// });
// client.addListener('message', (message) => {
// if (message.toString() == 'pong') {
// done();
// }
// });
// });

// it('execute method is not allowed', async () => {
// const body = {
// method: 'eth_getBlockByNumbers',
// params: ['0x548', true],
// id: 1,
// jsonrpc: '2.0',
// };

// client.on('open', async () => {
// client.send(JSON.stringify(body));
// });
// client.addListener('message', (message) => {
// const data = JSON.parse(message.toString());
// expect(data.method).toBe('eth_getBlockByNumbers');
// expect(data.response.error.message).toBe('method not allowed');
// });
// });

// it('executed method net_version', async () => {
// const body = {
// jsonrpc: '2.0',
// method: 'net_version',
// params: [],
// id: 1,
// };

// client.on('open', async () => {
// client.send(JSON.stringify(body));
// });
// client.addListener('message', (message) => {
// const data = JSON.parse(message.toString());
// expect(data.method).toBe('net_version');
// expect(data.response.result).toBe('12345');
// });
// });
});
116 changes: 116 additions & 0 deletions src/communicates/communicate.gateway.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,116 @@
import { Logger } from '@nestjs/common';
import { ConfigService } from '@nestjs/config';
import {
OnGatewayConnection,
OnGatewayDisconnect,
WebSocketGateway,
WebSocketServer,
} from '@nestjs/websockets';
import { Server } from 'ws';
import * as WebSocket from 'ws';
import { WebSocketService } from 'src/services/webSocket.sevice';
import {
CONNECTION_IS_CLOSED,
ESocketError,
INVALID_JSON_REQUEST,
METHOD_IS_NOT_ALLOWED,
} from 'src/constant/exception.constant';
import { TypeCheckService } from 'src/services';

@WebSocketGateway()
export class CommunicateGateway
implements OnGatewayConnection, OnGatewayDisconnect
{
@WebSocketServer() server: Server;
private logger: Logger = new Logger('AppGateway');
private allowedMethods: RegExp[];

constructor(
private readonly configService: ConfigService,
private readonly typeCheckService: TypeCheckService,
private readonly webSocketService: WebSocketService,
) {
this.allowedMethods = this.configService.get<RegExp[]>(
'allowedMethods',
) ?? [/^.*$/];
}

async handleDisconnect(): Promise<void> {
this.logger.log(`Client disconneted`);
this.webSocketService.close();
}

async handleConnection(client: WebSocket): Promise<void> {
this.logger.log(`Client connected`);
// connect to node's websocket
const url = this.configService.get<string>('nodeSocket')!;

Check warning on line 46 in src/communicates/communicate.gateway.ts

View workflow job for this annotation

GitHub Actions / check code

Forbidden non-null assertion
this.webSocketService.connect(url);

// listen to message from verse proxy websocket
client.on('message', (data) => {
const dataString = data.toString();
// for test connection
if (dataString == 'ping') {
return client.send('pong');
}

// check if server is connected to node or not
if (!this.webSocketService.isConnected()) {
client.send(CONNECTION_IS_CLOSED);
client.close();
}
try {
const jsonData = this.checkValidJson(dataString);
this.checkMethod(jsonData.method);
this.webSocketService.send(data.toString());
} catch (e) {
// if input not a valid json object or method is not then send message to client and close connect
switch (e.message) {
case ESocketError.INVALID_JSON_REQUEST:
client.send(INVALID_JSON_REQUEST);
break;
case ESocketError.METHOD_IS_NOT_ALLOWED:
client.send(METHOD_IS_NOT_ALLOWED);
break;
case ESocketError.CONNECTION_IS_CLOSED:
client.send(CONNECTION_IS_CLOSED);
break;
}
client.close();
}
});

// listen to message return from node's websocket
this.webSocketService.on((data: any) => {
const dataString = data.toString();
client.send(data);

// close connection if node's websocket return error
if (
this.typeCheckService.isJsonrpcErrorResponse(
JSON.parse(dataString.toString()),
)
) {
client.close();
}
});
}

checkValidJson(input: string) {
try {
const json = JSON.parse(input);
return json;
} catch {
throw new Error(ESocketError.INVALID_JSON_REQUEST);
}
}

checkMethod(method: string) {
const checkMethod = this.allowedMethods.some((allowedMethod) => {
return allowedMethod.test(method);
});
if (!checkMethod) {
throw new Error(ESocketError.METHOD_IS_NOT_ALLOWED);
}
}
}
3 changes: 3 additions & 0 deletions src/config/configuration.ts
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,9 @@ export default () => ({
/^eth_maxPriorityFeePerGas$/,
/^eth_feeHistory$/,
/^eth_.*Filter$/,
/^eth_unsubscribe$/,
/^eth_subscribe$/,
],
inheritHostHeader: true,
nodeSocket: process.env.NODE_SOCKET,
});
32 changes: 32 additions & 0 deletions src/constant/exception.constant.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
export const INVALID_JSON_REQUEST = `{
"jsonrpc": "2.0",
"id": null,
"error": {
"code": -32700,
"message": "invalid json format"
}
}`;

export const METHOD_IS_NOT_ALLOWED = `{
"jsonrpc": "2.0",
"id": null,
"error": {
"code": -32601,
"message": "method not allowed"
}
}`;

export const CONNECTION_IS_CLOSED = `{
"jsonrpc": "2.0",
"id": null,
"error": {
"code": -32601,
"message": "connection is closed"
}
}`;

export enum ESocketError {
INVALID_JSON_REQUEST = 'INVALID_JSON_REQUEST',
METHOD_IS_NOT_ALLOWED = 'METHOD_IS_NOT_ALLOWED',
CONNECTION_IS_CLOSED = 'CONNECTION_IS_CLOSED',
}
9 changes: 9 additions & 0 deletions src/entities/Error.ts
Original file line number Diff line number Diff line change
Expand Up @@ -6,3 +6,12 @@ export class JsonrpcError extends Error {
this.code = code;
}
}

export class WebsocketError extends Error {
code: number;

constructor(message: string, code: number) {
super(message);
this.code = code;
}
}
3 changes: 2 additions & 1 deletion src/main.ts
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@ import { AppModule } from 'src/app.module';
import { json } from 'body-parser';
import * as _cluster from 'cluster';
import { cpus } from 'os';
import { WsAdapter } from '@nestjs/platform-ws';

const cluster = _cluster as unknown as _cluster.Cluster;
let workerCount = process.env.CLUSTER_PROCESS
Expand All @@ -12,7 +13,7 @@ let workerCount = process.env.CLUSTER_PROCESS

async function bootstrap() {
const app = await NestFactory.create(AppModule);

app.useWebSocketAdapter(new WsAdapter(app));
app.use(
json({
limit: process.env.MAX_BODY_BYTE_SIZE
Expand Down
Loading
Loading