Skip to content
This repository has been archived by the owner on May 13, 2024. It is now read-only.

feat: add plugin macros related #32

Merged
merged 2 commits into from
Apr 18, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions src/SUMMARY.md
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,7 @@
# Architecture
- [rspack](./architecture/rspack/intro.md)
- [loader](./architecture/rspack/loader.md)
- [plugin](./architecture/rspack/plugin.md)
- [webpack](./architecture/webpack/intro.md)
- [loader](./architecture/webpack/loader.md)
- [dependency](./architecture/webpack/dependency.md)
Expand Down
4 changes: 3 additions & 1 deletion src/architecture/rspack/intro.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,4 +2,6 @@
This is the architecture of current rspack implementation

# Table of Contents
[loader](./loader.md)

- [loader](./loader.md)
- [plugin](./plugin.md)
58 changes: 58 additions & 0 deletions src/architecture/rspack/plugin.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,58 @@
# How to write a builtin plugin

Builtin plugin uses [rspack_macros](https://github.com/web-infra-dev/rspack/tree/7cc39cc4bb6f73791a5bcb175137ffd84b105da5/crates/rspack_macros) to help you avoid writing boilerplate code, you can use [cargo-expand](https://github.com/dtolnay/cargo-expand) or [rust-analyzer expand macro](https://rust-analyzer.github.io/manual.html#expand-macro-recursively) to checkout the expanded code, and for developing/testing these macro, you can starts with [rspack_macros_test](https://github.com/web-infra-dev/rspack/tree/7cc39cc4bb6f73791a5bcb175137ffd84b105da5/crates/rspack_macros_test).

A simple example:

```rust
use rspack_hook::{plugin, plugin_hook};
use rspack_core::{Plugin, PluginContext, ApplyContext, CompilerOptions};
use rspack_core::CompilerCompilation;
use rspack_error::Result;

// define the plugin
#[plugin]
pub struct MyPlugin {
options: MyPluginOptions
}

// define the plugin hook
#[plugin_hook(CompilerCompilation for MuPlugin)]
async fn compilation(&self, compilation: &mut Compilation) -> Result<()> {
// do something...
}

// implement apply method for the plugin
impl Plugin for MyPlugin {
fn apply(&self, ctx: PluginContext<&mut ApplyContext>, _options: &mut CompilerOptions) -> Result<()> {
ctx.context.compiler_hooks.tap(compilation::new(self))
Ok(())
}
}
```

And here is [an example](https://github.com/web-infra-dev/rspack/blob/7cc39cc4bb6f73791a5bcb175137ffd84b105da5/crates/rspack_plugin_ignore/src/lib.rs).

If the hook you need is not defined yet, you can define it by `rspack_hook::define_hook`, `compiler.hooks.assetEmitted` for example:

```rust
// this will allow you define hook's arguments without limit
define_hook!(CompilerShouldEmit: AsyncSeriesBail(compilation: &mut Compilation) -> bool);
// ------------------ --------------- ----------------------------- -------
// hook name exec kind hook arguments return value (Result<Option<bool>>)

#[derive(Debug, Default)]
pub struct CompilerHooks {
// ...
// and add it here
pub asset_emitted: CompilerAssetEmittedHook,
}
```

There are 5 kinds of exec kind:

- AsyncSeries, return value is `Result<()>`
- AsyncSeriesBail, return value is `Result<Option<T>>`
- AsyncParallel, return value is `Result<()>`
- SyncSeries, return value is `Result<()>`
- SyncSeriesBail, return value is `Result<Option<T>>`
36 changes: 18 additions & 18 deletions src/architecture/webpack/dependency.md
Original file line number Diff line number Diff line change
Expand Up @@ -91,11 +91,11 @@ For plugins, you can access via `module.buildInfo.contextDependencies`.

### Duplicated module detection

Each module will have its own `identifier`, for `NormalModule`, you can find this in `NormalModule#identifier`. If the identifier will be duplicated if inserted in `this._module`, then webpack will directly skip the remaining build process. [[source]](https://github.com/webpack/webpack/blob/9fcaa243573005d6fdece9a3f8d89a0e8b399613/lib/Compilation.js#L1270-L1274)
Each module will have its own `identifier`, for `NormalModule`, you can find this in `NormalModule#identifier`. If the identifier will be duplicated if inserted in `this._module`, then webpack will directly skip the remaining build process. [\[source\]](https://github.com/webpack/webpack/blob/9fcaa243573005d6fdece9a3f8d89a0e8b399613/lib/Compilation.js#L1270-L1274)

Basically, an `NormalModule` identifier contains these parts:
1. `type` [`string`]: The module type of a module. If the type of the module is `javascript/auto`, this field can be omitted
2. `request` [`string`]: Request to the module. All loaders whether it's inline or matched by a config will be stringified. If _inline match resource_ exists, inline loaders will be executed before any normal-loaders after pre-loaders. A module with a different loader passed through will be treated as a different module regardless of its path.
1. `type` \[`string`\]: The module type of a module. If the type of the module is `javascript/auto`, this field can be omitted
2. `request` \[`string`\]: Request to the module. All loaders whether it's inline or matched by a config will be stringified. If _inline match resource_ exists, inline loaders will be executed before any normal-loaders after pre-loaders. A module with a different loader passed through will be treated as a different module regardless of its path.
3. `layer`: applied if provided


Expand All @@ -104,7 +104,7 @@ Basically, an `NormalModule` identifier contains these parts:

`getResolve` is a loader API on the `LoaderContext`. Loader developers can pass `dependencyType` to its `option` which indicates the category of the module dependency that will be created. Values like `esm` can be passed, then webpack will use type `esm` to resolve the dependency.

The resolved dependencies are automatically added to the current module. This is driven by the internal plugin system of `enhanced-resolve`. Internally, `enhanced-resolve` uses plugins to handle the dependency registration like `FileExistsPlugin` [[source]](https://github.com/webpack/enhanced-resolve/blob/e5ff68aef5ab43b8197e864181eda3912957c526/lib/FileExistsPlugin.js#L34-L54) to detect whether a file is located on the file system or will add this file to a list of `missingDependency` and report in respect of the running mode of webpack. The collecting end of Webpack is generated by the `getResolveContext` in `NormalModule` [[source]](https://github.com/webpack/webpack/blob/9fcaa243573005d6fdece9a3f8d89a0e8b399613/lib/NormalModule.js#L513-L524)
The resolved dependencies are automatically added to the current module. This is driven by the internal plugin system of `enhanced-resolve`. Internally, `enhanced-resolve` uses plugins to handle the dependency registration like `FileExistsPlugin` [\[source\]](https://github.com/webpack/enhanced-resolve/blob/e5ff68aef5ab43b8197e864181eda3912957c526/lib/FileExistsPlugin.js#L34-L54) to detect whether a file is located on the file system or will add this file to a list of `missingDependency` and report in respect of the running mode of webpack. The collecting end of Webpack is generated by the `getResolveContext` in `NormalModule` [\[source\]](https://github.com/webpack/webpack/blob/9fcaa243573005d6fdece9a3f8d89a0e8b399613/lib/NormalModule.js#L513-L524)



Expand Down Expand Up @@ -181,14 +181,14 @@ HarmonyImportSideEffectDependency {

#### ESM-related derived types

There are a few of *ModuleDependencies* introduced in ESM imports. A full list of each derived type can be reached at [[source]](https://github.com/webpack/webpack/blob/86a8bd9618c4677e94612ff7cbdf69affeba1268/lib/dependencies/HarmonyImportDependencyParserPlugin.js)
There are a few of *ModuleDependencies* introduced in ESM imports. A full list of each derived type can be reached at [\[source\]](https://github.com/webpack/webpack/blob/86a8bd9618c4677e94612ff7cbdf69affeba1268/lib/dependencies/HarmonyImportDependencyParserPlugin.js)


##### Import

**`HarmonyImportDependency`**

The basic type of harmony-related *module dependencies* are below. [[source]](https://github.com/webpack/webpack/blob/86a8bd9618c4677e94612ff7cbdf69affeba1268/lib/dependencies/HarmonyImportDependency.js#L51)
The basic type of harmony-related *module dependencies* are below. [\[source\]](https://github.com/webpack/webpack/blob/86a8bd9618c4677e94612ff7cbdf69affeba1268/lib/dependencies/HarmonyImportDependency.js#L51)

**`HarmonyImportSideEffectDependency`**

Expand Down Expand Up @@ -220,7 +220,7 @@ import { foo, bar } from "./module"
console.log(foo, bar)
```

Specifier will be mapped into a specifier dependency if and only if it is used. JavaScript parser will first tag each variable [[source]](https://github.com/webpack/webpack/blob/86a8bd9618c4677e94612ff7cbdf69affeba1268/lib/dependencies/HarmonyImportDependencyParserPlugin.js#L137), and then create corresponding dependencies on each reading of dependency. [[source]](https://github.com/webpack/webpack/blob/86a8bd9618c4677e94612ff7cbdf69affeba1268/lib/dependencies/HarmonyImportDependencyParserPlugin.js#L189) and finally be replaced to the generated `importVar`.
Specifier will be mapped into a specifier dependency if and only if it is used. JavaScript parser will first tag each variable [\[source\]](https://github.com/webpack/webpack/blob/86a8bd9618c4677e94612ff7cbdf69affeba1268/lib/dependencies/HarmonyImportDependencyParserPlugin.js#L137), and then create corresponding dependencies on each reading of dependency. [\[source\]](https://github.com/webpack/webpack/blob/86a8bd9618c4677e94612ff7cbdf69affeba1268/lib/dependencies/HarmonyImportDependencyParserPlugin.js#L189) and finally be replaced to the generated `importVar`.


##### Export(They are not module dependencies to be actual, but I placed here for convenience)
Expand Down Expand Up @@ -352,14 +352,14 @@ As you can see from the `Source` section above, there is another modification we

#### `Fragments`

Essentially, a [_fragment_](https://github.com/webpack/webpack/blob/9fcaa243573005d6fdece9a3f8d89a0e8b399613/lib/InitFragment.js) is a pair of code snippet that to be wrapped around each _module_ source. Note the wording "wrap", it could contain two parts `content` and `endContent` [[source]](https://github.com/webpack/webpack/blob/9fcaa243573005d6fdece9a3f8d89a0e8b399613/lib/InitFragment.js#L69). To make it more illustrative, see this:
Essentially, a [_fragment_](https://github.com/webpack/webpack/blob/9fcaa243573005d6fdece9a3f8d89a0e8b399613/lib/InitFragment.js) is a pair of code snippet that to be wrapped around each _module_ source. Note the wording "wrap", it could contain two parts `content` and `endContent` [\[source\]](https://github.com/webpack/webpack/blob/9fcaa243573005d6fdece9a3f8d89a0e8b399613/lib/InitFragment.js#L69). To make it more illustrative, see this:

<img width="390" alt="image" src="https://user-images.githubusercontent.com/10465670/190576169-43ac19c4-2783-46c3-9059-b64b1ff72c4e.png">

The order of the fragment comes from two parts:
1. The stage of a fragment: if the stage of two fragments is different, then it will be replaced corresponding to the order define by the stage
2. If two fragments share the same order, then it will be replaced in [position](https://github.com/webpack/webpack/blob/9fcaa243573005d6fdece9a3f8d89a0e8b399613/lib/InitFragment.js#L41) order.
[[source]](https://github.com/webpack/webpack/blob/9fcaa243573005d6fdece9a3f8d89a0e8b399613/lib/InitFragment.js#L153-L159)
[\[source\]](https://github.com/webpack/webpack/blob/9fcaa243573005d6fdece9a3f8d89a0e8b399613/lib/InitFragment.js#L153-L159)

**A real-world example**

Expand Down Expand Up @@ -393,7 +393,7 @@ parser.hooks.import.tap(
}
);
```
Webpack will create two dependencies `ConstDependency` and `HarmonyImportSideEffectDependency` while parsing [[source]](https://github.com/webpack/webpack/blob/9fcaa243573005d6fdece9a3f8d89a0e8b399613/lib/dependencies/HarmonyImportDependencyParserPlugin.js#L110-L132).
Webpack will create two dependencies `ConstDependency` and `HarmonyImportSideEffectDependency` while parsing [\[source\]](https://github.com/webpack/webpack/blob/9fcaa243573005d6fdece9a3f8d89a0e8b399613/lib/dependencies/HarmonyImportDependencyParserPlugin.js#L110-L132).

Let me focus on `HarmonyImportSideEffectDependency` more, since it uses `Fragment` to do some patch.

Expand All @@ -407,7 +407,7 @@ HarmonyImportSideEffectDependency.Template = class HarmonyImportSideEffectDepend
}
};
```
As you can see in its associated _template_ [[source]](https://github.com/webpack/webpack/blob/9fcaa243573005d6fdece9a3f8d89a0e8b399613/lib/dependencies/HarmonyImportSideEffectDependency.js#L59), the modification to the code is made via its superclass `HarmonyImportDependency.Template` [[source]](https://github.com/webpack/webpack/blob/9fcaa243573005d6fdece9a3f8d89a0e8b399613/lib/dependencies/HarmonyImportDependency.js#L244).
As you can see in its associated _template_ [\[source\]](https://github.com/webpack/webpack/blob/9fcaa243573005d6fdece9a3f8d89a0e8b399613/lib/dependencies/HarmonyImportSideEffectDependency.js#L59), the modification to the code is made via its superclass `HarmonyImportDependency.Template` [\[source\]](https://github.com/webpack/webpack/blob/9fcaa243573005d6fdece9a3f8d89a0e8b399613/lib/dependencies/HarmonyImportDependency.js#L244).

```js
// some code is omitted for cleaner demonstration
Expand Down Expand Up @@ -447,7 +447,7 @@ As you can see from the simplified source code above, the actual patch made to t
/* harmony import */ var _foo__WEBPACK_IMPORTED_MODULE_0__ = __webpack_require__(/*! ./foo */ "./src/foo.js"); //(1)
```

Note, the real require statement is generated via _initFragments_, `ConditionalInitFragment` to be specific. Don't be afraid of the naming, for more information you can see the (background)[https://github.com/webpack/webpack/pull/11802] of this _fragment_, which let's webpack to change it from `InitFragment` to `ConditionalInitFragment`.
Note, the real require statement is generated via _initFragments_, `ConditionalInitFragment` to be specific. Don't be afraid of the naming, for more information you can see the [background](https://github.com/webpack/webpack/pull/11802) of this _fragment_, which let's webpack to change it from `InitFragment` to `ConditionalInitFragment`.

**How does webpack solve the compatibility issue?**

Expand All @@ -471,7 +471,7 @@ Finally, also known as the third iteration of collection, Webpack hoists `runtim

![image-20220919174132772](https://raw.githubusercontent.com/h-a-n-a/static/main/2022/09/upgit_20220919_1663580492.png)

The referenced source code you can be found it [here](https://github.com/webpack/webpack/blob/9fcaa243573005d6fdece9a3f8d89a0e8b399613/lib/Compilation.js#L3379) and these steps are basically done in `processRuntimeRequirements`. This let me recall the linking procedure of a rollup-like bundler. Anyway, after this procedure, we can finally generate _runtime modules_. Actually, I lied here, huge thanks to the hook system of Webpack, the creation of _runtime modules_ is done in this method via calls to `runtimeRequirementInTree`[[source]](https://github.com/webpack/webpack/blob/9fcaa243573005d6fdece9a3f8d89a0e8b399613/lib/Compilation.js#L3498). No doubt, this is all done in the `seal` step. After that, webpack will process each chunk and create a few code generation jobs, and finally, emit assets.
The referenced source code you can be found it [here](https://github.com/webpack/webpack/blob/9fcaa243573005d6fdece9a3f8d89a0e8b399613/lib/Compilation.js#L3379) and these steps are basically done in `processRuntimeRequirements`. This let me recall the linking procedure of a rollup-like bundler. Anyway, after this procedure, we can finally generate _runtime modules_. Actually, I lied here, huge thanks to the hook system of Webpack, the creation of _runtime modules_ is done in this method via calls to `runtimeRequirementInTree`[\[source\]](https://github.com/webpack/webpack/blob/9fcaa243573005d6fdece9a3f8d89a0e8b399613/lib/Compilation.js#L3498). No doubt, this is all done in the `seal` step. After that, webpack will process each chunk and create a few code generation jobs, and finally, emit assets.



Expand All @@ -489,9 +489,9 @@ if (module.hot) {
}
```

Webpack will replace expressions like `module.hot` and `module.hot.accept`, etc with `ConstDependency` as the *presentationalDependency* as I previously talked about. [[source]](https://github.com/webpack/webpack/blob/9fcaa243573005d6fdece9a3f8d89a0e8b399613/lib/HotModuleReplacementPlugin.js#L97-L101)
Webpack will replace expressions like `module.hot` and `module.hot.accept`, etc with `ConstDependency` as the *presentationalDependency* as I previously talked about. [\[source\]](https://github.com/webpack/webpack/blob/9fcaa243573005d6fdece9a3f8d89a0e8b399613/lib/HotModuleReplacementPlugin.js#L97-L101)

With the help of a simple expression replacement is not enough, the plugin also introduce additional runtime modules for each entries. [[source]](https://github.com/webpack/webpack/blob/9fcaa243573005d6fdece9a3f8d89a0e8b399613/lib/HotModuleReplacementPlugin.js#L736-L748)
With the help of a simple expression replacement is not enough, the plugin also introduce additional runtime modules for each entries. [\[source\]](https://github.com/webpack/webpack/blob/9fcaa243573005d6fdece9a3f8d89a0e8b399613/lib/HotModuleReplacementPlugin.js#L736-L748)

The plugin is quite complicated, and you should definitely checkout what it actually does, but for things related to dependency, it's enough.

Expand All @@ -503,7 +503,7 @@ The plugin is quite complicated, and you should definitely checkout what it actu

### Constant folding

> The logic is defined in ConstPlugin : [[source]](https://github.com/webpack/webpack/blob/9fcaa243573005d6fdece9a3f8d89a0e8b399613/lib/ConstPlugin.js#L135)
> The logic is defined in ConstPlugin : [\[source\]](https://github.com/webpack/webpack/blob/9fcaa243573005d6fdece9a3f8d89a0e8b399613/lib/ConstPlugin.js#L135)


_Constant folding_ is a technique that used as an optimization for optimization. For example:
Expand Down Expand Up @@ -566,7 +566,7 @@ module.exports = {

![image-20220919190925073](https://raw.githubusercontent.com/h-a-n-a/static/main/2022/09/upgit_20220919_1663585765.png)

As you can see from the red square, the `initFragment` is generated based on the usage of the exported symbol in the `HarmonyExportSpecifierDependency` [[source]](https://github.com/webpack/webpack/blob/9fcaa243573005d6fdece9a3f8d89a0e8b399613/lib/dependencies/HarmonyExportSpecifierDependency.js#L91-L107)
As you can see from the red square, the `initFragment` is generated based on the usage of the exported symbol in the `HarmonyExportSpecifierDependency` [\[source\]](https://github.com/webpack/webpack/blob/9fcaa243573005d6fdece9a3f8d89a0e8b399613/lib/dependencies/HarmonyExportSpecifierDependency.js#L91-L107)

If `foo` is used in the graph, then the generated result will be this:

Expand All @@ -577,7 +577,7 @@ If `foo` is used in the graph, then the generated result will be this:
const foo = "foo";
```

In the example above, the `foo` is not used, so it will be excluded in the code generation of the template of `HarmonyExportSpecifierDependency` and it will be dead-code-eliminated in later steps. For terser plugin, it eliminates all unreachable code in `processAssets` [[source]](https://github.com/webpack-contrib/terser-webpack-plugin/blob/580f59c5d223a31c4a9c658a6f9bb1e59b3defa6/src/index.js#L836).
In the example above, the `foo` is not used, so it will be excluded in the code generation of the template of `HarmonyExportSpecifierDependency` and it will be dead-code-eliminated in later steps. For terser plugin, it eliminates all unreachable code in `processAssets` [\[source\]](https://github.com/webpack-contrib/terser-webpack-plugin/blob/580f59c5d223a31c4a9c658a6f9bb1e59b3defa6/src/index.js#L836).



Expand Down
5 changes: 3 additions & 2 deletions src/architecture/webpack/intro.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,5 +3,6 @@
This is the architecture of webpack implementation

# Table of Contents
[loader](./loader.md)
[dependency](./dependency.md)

- [loader](./loader.md)
- [dependency](./dependency.md)
Loading
Loading