Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: load wasm #1705

Open
wants to merge 14 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 2 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 14 additions & 0 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

1 change: 1 addition & 0 deletions crates/mako/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,7 @@ percent-encoding = { version = "2.3.1" }
serde = { workspace = true }
serde_json = { workspace = true }
url = { version = "2.5.0" }
wasmparser = "0.207.0"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codebase verification

建议更新 wasmparser 依赖版本

当前使用的版本 (0.207.0) 已过时。最新的稳定版本是 0.220.0,建议:

  1. 更新到最新版本
  2. 使用版本范围声明以自动获取补丁更新:wasmparser = "^0.220.0"

安全检查显示该包没有已知的安全漏洞。

🔗 Analysis chain

建议检查 wasmparser 的版本选择

建议考虑以下几点:

  1. 使用版本范围声明(例如 ^0.207.0)来自动接收补丁版本的更新
  2. 确认当前选择的版本是否为最新的稳定版本

运行以下脚本来验证版本信息:

🏁 Scripts executed

The following scripts were executed for the analysis:

Script:

#!/bin/bash
# 检查 wasmparser 包的最新版本和安全公告

# 从 crates.io 获取最新版本信息
curl -s https://crates.io/api/v1/crates/wasmparser | jq -r '.crate.max_stable_version'

# 检查安全公告
gh api graphql -f query='
{
  securityVulnerabilities(first: 5, ecosystem: RUST, package: "wasmparser") {
    nodes {
      advisory {
        summary
        severity
        publishedAt
      }
      vulnerableVersionRange
      firstPatchedVersion {
        identifier
      }
    }
  }
}'

Length of output: 439


swc_core = { workspace = true, features = [
"base",
Expand Down
86 changes: 85 additions & 1 deletion crates/mako/src/plugins/wasm_runtime.rs
Original file line number Diff line number Diff line change
@@ -1,9 +1,14 @@
use std::collections::HashMap;
use std::fs::File;
use std::io::Read;
use std::sync::Arc;

use anyhow;
use wasmparser::{Import, Parser, Payload};

use crate::ast::file::{Content, JsContent};
use crate::compiler::Context;
use crate::plugin::Plugin;
use crate::plugin::{Plugin, PluginLoadParam};

pub struct WasmRuntimePlugin {}

Expand All @@ -27,4 +32,83 @@ impl Plugin for WasmRuntimePlugin {
Ok(vec![])
}
}

fn load(
&self,
param: &PluginLoadParam,
_context: &Arc<Context>,
) -> anyhow::Result<Option<Content>> {
let file = param.file;
if file.path.to_string_lossy().ends_with(".wasm") {
let final_file_name = format!(
"{}.{}.{}",
file.get_file_stem(),
file.get_content_hash()?,
file.extname
);
let origin_path = file.pathname.to_string_lossy().to_string();
_context.emit_assets(origin_path, final_file_name.clone());

let mut buffer = Vec::new();
File::open(file.path.as_path())?.read_to_end(&mut buffer)?;
// Parse wasm file to get imports
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🛠️ Refactor suggestion

建议改进错误处理和内存管理

当前实现存在以下需要注意的地方:

  1. 文件操作缺少详细的错误处理,建议添加具体的错误信息
  2. 将整个WASM文件读入内存可能会对大文件造成问题

建议修改如下:

- File::open(file.path.as_path())?.read_to_end(&mut buffer)?;
+ File::open(&file.path)
+     .map_err(|e| anyhow::anyhow!("无法打开WASM文件 {}: {}", file.path.display(), e))?
+     .read_to_end(&mut buffer)
+     .map_err(|e| anyhow::anyhow!("读取WASM文件失败: {}", e))?;

对于大文件,可以考虑使用流式处理或设置最大文件大小限制。

📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
let file = param.file;
if file.path.to_string_lossy().ends_with(".wasm") {
let final_file_name = format!(
"{}.{}.{}",
file.get_file_stem(),
file.get_content_hash()?,
file.extname
);
let origin_path = file.pathname.to_string_lossy().to_string();
_context.emit_assets(origin_path, final_file_name.clone());
let mut buffer = Vec::new();
File::open(file.path.as_path())?.read_to_end(&mut buffer)?;
// Parse wasm file to get imports
let file = param.file;
if file.path.to_string_lossy().ends_with(".wasm") {
let final_file_name = format!(
"{}.{}.{}",
file.get_file_stem(),
file.get_content_hash()?,
file.extname
);
let origin_path = file.pathname.to_string_lossy().to_string();
_context.emit_assets(origin_path, final_file_name.clone());
let mut buffer = Vec::new();
File::open(&file.path)
.map_err(|e| anyhow::anyhow!("无法打开WASM文件 {}: {}", file.path.display(), e))?
.read_to_end(&mut buffer)
.map_err(|e| anyhow::anyhow!("读取WASM文件失败: {}", e))?;
// Parse wasm file to get imports

let mut import_objs_map: HashMap<&str, Vec<String>> = HashMap::new();
for payload in Parser::new(0).parse_all(&buffer) {
if let Ok(Payload::ImportSection(imports)) = payload {
for import in imports {
if let Ok(Import { module, name, ty }) = import {
if let Some(import_obj) = import_objs_map.get_mut(module) {
import_obj.push(name.to_string());
} else {
import_objs_map.insert(module, vec![name.to_string()]);
}
}
}
}
}

let mut js_imports_str = String::new();
let mut import_objs_str = String::new();

for (index, (key, value)) in import_objs_map.iter().enumerate() {
js_imports_str.push_str(&format!(
"import * as module{module_idx} from \"{module}\";\n",
module_idx = index,
module = key
));

import_objs_str.push_str(&format!(
"\"{module}\": {{ {names} }}",
module = key,
names = value
.iter()
.map(|name| format!("{}: module{}.{}", name, index, name))
.collect::<Vec<String>>()
.join(", ")
));
}

let mut content = String::new();
content.push_str(&js_imports_str);

if import_objs_str.is_empty() {
content.push_str(&format!(
"module.exports = require._interopreRequireWasm(exports, \"{}\")",
final_file_name
));
} else {
content.push_str(&format!(
"module.exports = require._interopreRequireWasm(exports, \"{}\", {{{}}})",
xusd320 marked this conversation as resolved.
Show resolved Hide resolved
final_file_name, import_objs_str
));
}

return Ok(Some(Content::Js(JsContent {
content,
..Default::default()
})));
}

Ok(None)
}
}
5 changes: 5 additions & 0 deletions examples/import-resources/index.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@ import toml, { title } from './index.toml';
import xml from './index.xml';
import yaml, { pi } from './index.yaml';
import MailchimpUnsplash from './mailchimp-unsplash.jpg';
import * as wasm from './minus-wasm-pack';

const num1 = 10;
const num2 = 20;
Expand All @@ -32,6 +33,10 @@ function App() {
<h2>
Test import .wasm file async: {num1} + {num2} = {sum}
</h2>
<h2>
Test import .wasm file(generated by wasm-pack) async: {num1} - {num2} ={' '}
{wasm.minus(num1, num2)}
</h2>
<div>
<h2>Test import .toml file</h2>
<pre>{JSON.stringify(toml, null, 2)}</pre>
Expand Down
15 changes: 15 additions & 0 deletions examples/import-resources/minus-wasm-pack/index.d.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
/* tslint:disable */
/* eslint-disable */
/**
*/
export function greet(): void;
/**
* @param {string} name
*/
export function greet2(name: string): void;
/**
* @param {number} a
* @param {number} b
* @returns {number}
*/
export function minus(a: number, b: number): number;
4 changes: 4 additions & 0 deletions examples/import-resources/minus-wasm-pack/index.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
import { __wbg_set_wasm } from './index_bg.js';
import * as wasm from './index_bg.wasm';
__wbg_set_wasm(wasm);
export * from './index_bg.js';
133 changes: 133 additions & 0 deletions examples/import-resources/minus-wasm-pack/index_bg.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,133 @@
let wasm;
export function __wbg_set_wasm(val) {
wasm = val;
}

const lTextDecoder =
typeof TextDecoder === 'undefined'
? (0, module.require)('util').TextDecoder
: TextDecoder;

let cachedTextDecoder = new lTextDecoder('utf-8', {
ignoreBOM: true,
fatal: true,
});

cachedTextDecoder.decode();

let cachedUint8ArrayMemory0 = null;

function getUint8ArrayMemory0() {
if (
cachedUint8ArrayMemory0 === null ||
cachedUint8ArrayMemory0.byteLength === 0
) {
cachedUint8ArrayMemory0 = new Uint8Array(wasm.memory.buffer);
}
return cachedUint8ArrayMemory0;
}
xusd320 marked this conversation as resolved.
Show resolved Hide resolved

function getStringFromWasm0(ptr, len) {
ptr = ptr >>> 0;
return cachedTextDecoder.decode(
getUint8ArrayMemory0().subarray(ptr, ptr + len),
);
}
/**
*/
export function greet() {
wasm.greet();
}

let WASM_VECTOR_LEN = 0;

const lTextEncoder =
typeof TextEncoder === 'undefined'
? (0, module.require)('util').TextEncoder
: TextEncoder;

let cachedTextEncoder = new lTextEncoder('utf-8');

const encodeString =
typeof cachedTextEncoder.encodeInto === 'function'
? function (arg, view) {
return cachedTextEncoder.encodeInto(arg, view);
}
: function (arg, view) {
const buf = cachedTextEncoder.encode(arg);
view.set(buf);
return {
read: arg.length,
written: buf.length,
};
};

function passStringToWasm0(arg, malloc, realloc) {
if (realloc === undefined) {
const buf = cachedTextEncoder.encode(arg);
const ptr = malloc(buf.length, 1) >>> 0;
getUint8ArrayMemory0()
.subarray(ptr, ptr + buf.length)
.set(buf);
WASM_VECTOR_LEN = buf.length;
return ptr;
}

let len = arg.length;
let ptr = malloc(len, 1) >>> 0;

const mem = getUint8ArrayMemory0();

let offset = 0;

for (; offset < len; offset++) {
const code = arg.charCodeAt(offset);
if (code > 0x7f) break;
mem[ptr + offset] = code;
}

if (offset !== len) {
if (offset !== 0) {
arg = arg.slice(offset);
}
ptr = realloc(ptr, len, (len = offset + arg.length * 3), 1) >>> 0;
const view = getUint8ArrayMemory0().subarray(ptr + offset, ptr + len);
const ret = encodeString(arg, view);

offset += ret.written;
ptr = realloc(ptr, len, offset, 1) >>> 0;
xusd320 marked this conversation as resolved.
Show resolved Hide resolved
}

WASM_VECTOR_LEN = offset;
return ptr;
}
xusd320 marked this conversation as resolved.
Show resolved Hide resolved
/**
* @param {string} name
*/
export function greet2(name) {
const ptr0 = passStringToWasm0(
name,
wasm.__wbindgen_malloc,
wasm.__wbindgen_realloc,
);
const len0 = WASM_VECTOR_LEN;
wasm.greet2(ptr0, len0);
}

/**
* @param {number} a
* @param {number} b
* @returns {number}
*/
export function minus(a, b) {
const ret = wasm.minus(a, b);
return ret;
}

export function __wbg_alert_f837f172b2a24942(arg0, arg1) {
alert(getStringFromWasm0(arg0, arg1));
}

export function __wbg_prompt_ec584a06a1c7c28b(arg0, arg1) {
prompt(getStringFromWasm0(arg0, arg1));
}
Binary file not shown.
13 changes: 13 additions & 0 deletions examples/import-resources/minus-wasm-pack/index_bg.wasm.d.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
/* tslint:disable */
/* eslint-disable */
export const memory: WebAssembly.Memory;
export function greet2(a: number, b: number): void;
export function minus(a: number, b: number): number;
export function greet(): void;
export function __wbindgen_malloc(a: number, b: number): number;
export function __wbindgen_realloc(
a: number,
b: number,
c: number,
d: number,
): number;