Skip to content

Commit

Permalink
fix: CLI should accept job args
Browse files Browse the repository at this point in the history
  • Loading branch information
fi3ework committed Jun 17, 2024
1 parent 4d15869 commit 1ac714a
Show file tree
Hide file tree
Showing 3 changed files with 52 additions and 28 deletions.
68 changes: 42 additions & 26 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,43 +1,59 @@
# Rspack Benchmark

This repository is used to monitor rspack performance.
This repository is used to monitor Rspack performance.

## Usage

You can use the scripts in the `bin` directory to prepare and run benchmark.
You can use the scripts in the `bin` directory to prepare and run benchmark. All interactions are of the form

- `node bin/build-rspack.js [remote] [branch]`
```
node bin/cli.js [command] [args]
```

Clone and build [rspack](https://github.com/web-infra-dev/rspack) in the folder which defined by the environment variable of `RSPACK_DIR` or `<project_root>/.rspack`. You can set target branch with the parameter. eg.
If no command is specified, then `build`, `bench` and `compare` commands will all be ran by default.

```bash
node bin/build-rspack.js # use the main branch to build
node bin/build-rspack.js origin main # use the main branch to build
node bin/build-rspack.js origin pull/1000/head # use the pull request with index 1000 to build
```
### Commands

- `node bin/bench.js [benchmarkNames...]`
#### build

Run benchmark with rspack and write the output to `output` folder. You can configure the environment variable of `RSPACK_DIR` to set the rspack project path, and it will use the rspack from the `.rspack` folder by default. eg.
```
node bin/cli.js build [--ref <ref>]
```

Clone and build [Rspack](https://github.com/web-infra-dev/rspack) in the folder which defined by the environment variable of `RSPACK_DIR` or `<project_root>/.rspack`. You can set Git ref with the parameter, e.g.:

```bash
node bin/bench.js # run all benchmarks
node bin/bench.js 10000_development-mode 10000_production-mode # run benchmarks named 10000_development-mode and 10000_production-mode
RSPACK_DIR=<your-rspack-path> node bin/bench.js 10000_development-mode_hmr # set the rspack command path, and run 10000_development-mode_hmr
node bin/cli.js build # use the main branch to build
node bin/cli.js build --ref main # use the main branch to build
node bin/cli.js build --ref pull/1000/head # use the pull request with index 1000 to build
```

- `node bin/compare-bench.js <baseDate> <currentDate>`
#### bench

```
node bin/cli.js bench [--job <benchmark-names> ...]
```

Compare and print the difference between `<baseDate>` and `<currentDate>`. The parameter has three types, `current` will use the data from `output` folder. `latest` will use the latest data from `data` branch. A date string like `YYYY-MM-DD` will use the data of that day from `data` branch. eg.
Run benchmark with Rspack and write the output to `output` folder. You can configure the environment variable of `RSPACK_DIR` to set the Rspack project path, and it will use the Rspack from the `.rspack` folder by default. eg.

```bash
node bin/compare-bench.js current latest # use output data as base, and latest data as current
node bin/compare-bench.js latest 2023-08-17 # use latest data as base, and the data of 2023-08-17 as current
node bin/cli.js bench # run all benchmarks
node bin/cli.js bench --job 10000_development-mode --job 10000_production-mode # run benchmarks named 10000_development-mode and 10000_production-mode
RSPACK_DIR=<your-rspack-path> node bin/cli.js bench --job 10000_development-mode_hmr # set the rspack command path, and run 10000_development-mode_hmr
```

#### compare

```
node bin/cli.js compare [--base <baseDate>] [--current <currentDate>]
```

- `node bin/upload.js`
Compare and print the difference between `<baseDate>` and `<currentDate>`. The parameter has three types, `"current"` will use the data from `output` folder. `"latest"` will use the latest data from `data` branch. A date string like `YYYY-MM-DD` will use the data of that day from `data` branch. eg.

Clone the data branch to `.data` folder, copy `output` folder into it and push it to the remote.
```bash
node bin/cli.js compare --base current --current latest # use output data as base, and latest data as current
node bin/cli.js compare --base latest --current 2023-08-17 # use latest data as base, and the data of 2023-08-17 as current
```

## Glossary

Expand All @@ -47,14 +63,14 @@ Benchmark name is a string containing the case name and the addon names. A bench

#### Case

The benchmark case is the rspack project in `cases` folder. It must contain `rspack.config.js` and `hmr.js`, the first one is the default config for rspack, the next one is used to tell benchmark how to change file when hmr.
The benchmark case is the Rspack project in `cases` folder. It must contain `rspack.config.js` and `hmr.js`, the first one is the default config for Rspack, the next one is used to tell benchmark how to change file when hmr.

- `10000` is a project with 10000 modules
- `threejs` is a copy of three js

#### Addon

The addon is used to change rspack configuration and benchmark parameters. All addons are registered in `lib/addons`.
The addon is used to change Rspack configuration and benchmark parameters. All addons are registered in `lib/addons`.

- `development-mode` is used to set the development mode
- `production-mode` is used to set the production mode
Expand Down Expand Up @@ -161,13 +177,13 @@ interface Addon {

1. move your project into the `cases` folder
2. move the project dependencies to global package.json
3. add `rspack.config.js` to make the project runnable by rspack
3. add `rspack.config.js` to make the project runnable by Rspack
4. add `hmr.js` to make the project support hmr changes
5. try run `RSPACK_DIR=<your-rspack-path> node bin/bench.js <your case>_<your addons>` to test
5. try run `RSPACK_DIR=<your-rspack-path> node bin/cli.js bench <your case>_<your addons>` to test

## How to create a addon

1. create your addons in `lib/addons` with kebab case like "a-b-c.js"
2. export default a class that extends `lib/addons/common` and implement the hooks you want to listen
3. try run `RSPACK_DIR=<your-rspack-path> node bin/bench.js <case>_<your addons>` to test
4. if you want to run some benchmarkName by default, you can add it to `defaultBenchmarkNames` in `bin/bench.js`
3. try run `RSPACK_DIR=<your-rspack-path> node bin/cli.js bench <case>_<your addons>` to test
4. if you want to run some benchmark name by default, you can add it to `jobs` field in `./bench.config.js`.
8 changes: 6 additions & 2 deletions bin/cli.js
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,10 @@ const cli = meow({
type: "boolean",
default: true
},

job: {
type: "string",
isMultiple: true
},
shard: {
type: "string",
default: "1/1"
Expand All @@ -57,6 +60,7 @@ const {
repository,
ref,

job: _job,
binding,
js,

Expand All @@ -71,7 +75,7 @@ const cwd = process.cwd();
const configPath = join(process.cwd(), "bench.config.js");
const config = (await import(configPath)).default;

const jobs = config.jobs ?? [];
const jobs = (_job.length === 0 ? undefined : _job) ?? config.jobs ?? [];
const rspackDirectory = config.rspackDirectory ?? join(cwd, ".rspack");
const benchmarkDirectory = config.benchmarkDirectory ?? join(cwd, "output");

Expand Down
4 changes: 4 additions & 0 deletions pnpm-lock.yaml

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

0 comments on commit 1ac714a

Please sign in to comment.