Skip to content

Commit

Permalink
build SEA binaries with github action
Browse files Browse the repository at this point in the history
this change
- adds a github action to build SEA binaries for linux-x64, windows-x64,
  mac-x64, and mac-arm64
- adds documentation for the SEA build
  • Loading branch information
echo-bravo-yahoo committed Oct 25, 2024
1 parent 66a6350 commit b77aab3
Show file tree
Hide file tree
Showing 9 changed files with 159 additions and 15 deletions.
59 changes: 59 additions & 0 deletions .github/workflows/build-binaries.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,59 @@
name: Build binaries
on:
push:
branches: [build]

jobs:
build-binaries:
name: Build binaries

strategy:
matrix:
runner: [macos-13, macos-latest, ubuntu-latest, windows-latest]
node: [22.x]
include:
- runner: macos-13
os: mac
arch: x64
- runner: macos-latest
os: mac
arch: arm
- runner: ubuntu-latest
os: linux
arch: x64
- runner: windows-latest
os: windows
arch: x64

runs-on: ${{ matrix.runner }}
steps:
- uses: actions/checkout@v4
- uses: actions/setup-node@v4
with:
node-version: ${{ matrix.node }}
- run: npm ci
- name: Build binary
run: npm run build
- uses: actions/upload-artifact@v4
with:
# Name of the artifact to upload.
name: fauna-shell-${{ matrix.os }}-${{ matrix.arch }}-${{ matrix.node }}

# A file, directory or wildcard pattern that describes what to upload
path: ${{ matrix.os == 'windows' && 'dist\fauna.exe' || 'dist/fauna' }}

# Fail the action with an error message if no files are found at the path.
if-no-files-found: error

# Duration after which artifact will expire in days. 0 means use the repository's default retention.
retention-days: 0

# The level of compression for Zlib to be applied to the artifact archive from 0 (none) to 9 (most).
compression-level: 6

# Deletes any artifact with a matching name before a new one is uploaded.
# Does not fail if the artifact does not exist.
overwrite: true

# Don't upload hidden files in the provided path.
include-hidden-files: false
8 changes: 8 additions & 0 deletions DEV-README.md
Original file line number Diff line number Diff line change
@@ -1,15 +1,23 @@
### Application versions

This project has 3 runnable entrypoints (a raw ESM one, a built CJS one, and an SEA one). You can read more about them [here](./sea/README.md).

### General style guidelines

- Prefer to throw errors instead of exiting the process. Exit is harder to mock well in tests, and the global error-handler in `src/cli.mjs` should already do verbosity-aware error-handling. You can request a specific exit code by attaching an `exitCode` property to your error before throwing it. The error-handling has a series of tests in `yargs-test/general-cli.mjs`; if you find a case where throwing results in bad output to the user, replicate that case in a test suite.
- Prefer to re-throw an existing error after modifying its message over catching and throwing a newly-constructed error. The `exitCode` and `stack` properties on the existing error are worth keeping.

#### Testing guidelines

- Prefer to mock the "far" edges of the application - methods on `fs`, complex async libraries (e.g., `http#Server`), `fetch`. This results in the test code traversing all of the CLI's business logic, but not interacting with error-prone external resources like disk, network, port availability, etc. `sinon` records all calls to a mock, and allows asserting against them. Use this if, e.g., your business logic calls `fetch` multiple times.
- ~~Prefer to run local tests in watch mode with (e.g., with `yarn local-test`) while developing.~~ This is currently broken.
- Use debug logs to output the shape of objects (especially network responses) to determine how to structure mocks. For instance, to get a quick mock for a network request caused by `fauna schema status ...`, set the situation up and run `fauna schema status ... --verbosity 5`. You can then use the logged objects as mocks.

#### Debugging strategies

- Fetch is not particularly amenable to debugging, but if you need to see the raw requests being made, open a netcat server locally (`nc -l 8080`) and then change the code to call the netcat server, either by passing the flag `--url http://localhost:8080` or by editing the code.
- This project has debug logging with a somewhat configurable logging strategy. To use it, provide either:
- `--verbose-component foo`, which logs all debug info for the component `foo`. Because it's an array, you can specify it multiple times. To see the available components, look at the help or tab completion.
- `--verbosity level`, where `level` is a number from 0 (no debug logging) to 5 (all debug logging) for all components.
- To investigate why a sinon stub is failing in a way you don't expect, you can log out `stub.args`, which is an array of the arguments provided each time the stub was called. This can be particularly helpful for assertions where the error message is hard to read or involves objects that don't stringify correctly (like FormData).
- To trigger [SEA builds](./sea/README.md) on all supported OS/architecture combinations, make changes and push commits to the `build` branch in github. This will [trigger github actions](./.github/workflows/build-binaries.yml) that build the SEA binaries.
2 changes: 1 addition & 1 deletion package.json
Original file line number Diff line number Diff line change
Expand Up @@ -69,7 +69,7 @@
"test:local": "mocha --recursive ./test --require ./test/mocha-root-hooks.mjs",
"build": "npm run build:app && npm run build:sea",
"build:app": "esbuild --bundle ./src/user-entrypoint.mjs --platform=node --outfile=./dist/cli.cjs --format=cjs --inject:./sea/import-meta-url.js --define:import.meta.url=importMetaUrl",
"build:sea": "./sea/build-sea.sh",
"build:sea": "node ./sea/build.cjs",
"format": "prettier -w src test package.json prettier.config.js eslint.config.mjs"
},
"husky": {
Expand Down
21 changes: 21 additions & 0 deletions sea/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
### SEA (single executable application)

This directory contains the infrastructure for building `fauna-shell` as a single executable application. You can find the docs for SEA [here](https://nodejs.org/docs/latest-v22.x/api/single-executable-applications.html#single-executable-applications); since this feature is experimental, make sure you're looking at the same nodeJS version as the project uses; there will be breaking changes across nodeJS versions.

The process generally looks like this:

1. A developer (or CI) runs [npm run build](../package.json).
2. `build:app` runs `eslint` to build the ES module CLI into a single-file CJS module with its dependencies' inlined. There are a few wrinkles here with `import.meta.url` and `__dirname`, but it's otherwise fairly straight-forward. This is what `./sea/import-meta-url.js` is for.
3. `build:sea` runs `./sea/build.cjs`. This nodeJS script detects the OS and builds an SEA for that OS. One of the inputs to this process is `./sea/config.json`, which specifies some paths and settings for the resulting build. We could optimize our builds here by enabling `useSnapshot` and `useCodeCache`, but that's likely not worth the effort until we have a (basic) perf benchmark in place.

### Versions of the CLI

1. The raw (runnable!) ESM CLI can be invoked by `./src/user-entrypoint.mjs <command> [subcommand] [args]`.
2. The built CJS CLI can be invoked by `./dist/cli.cjs <command> [subcommand] [args]`.
3. The SEA CLI can be invoked by `./dist/fauna <command> [subcommand] [args]`.

### Differences between versions

_All 3 versions should be runnable and behave the same, with these exceptions:_

- Currently, no exceptions.
8 changes: 0 additions & 8 deletions sea/build-sea.sh

This file was deleted.

59 changes: 59 additions & 0 deletions sea/build.cjs
Original file line number Diff line number Diff line change
@@ -0,0 +1,59 @@
/* eslint-disable no-console */

const os = require("node:os");
const { execSync: execSyncActual } = require("node:child_process");
const fs = require("node:fs");
const execSync = (command, options) =>
execSyncActual(command, { ...options, encoding: "utf-8" });

const platform = os.platform();
const arch = os.arch();
if (platform === "linux") {
console.log(`Building for Linux (${arch})...`);
buildSEAForLinux();
} else if (platform === "darwin") {
console.log(`Building for Mac (${arch})...`);
buildSEAForMac();
} else if (platform === "win32") {
console.log(`Building for Windows (${arch})...`);
buildSEAForWindows();
} else {
throw new Error(`No build configured for platform ${platform} and arch ${arch}!`);
}

function buildSEAForLinux() {
execSync("node --experimental-sea-config ./sea/config.json");
fs.copyFileSync(process.execPath, "./dist/fauna");
execSync(
"npx postject ./dist/fauna NODE_SEA_BLOB ./dist/sea.blob \
--sentinel-fuse NODE_SEA_FUSE_fce680ab2cc467b6e072b8b5df1996b2",
);
execSync("chmod +x ./dist/fauna");
}

function buildSEAForMac() {
execSync("node --experimental-sea-config ./sea/config.json");
fs.copyFileSync(process.execPath, "./dist/fauna");
execSync("codesign --remove-signature ./dist/fauna");
execSync(
"npx postject ./dist/fauna NODE_SEA_BLOB ./dist/sea.blob \
--sentinel-fuse NODE_SEA_FUSE_fce680ab2cc467b6e072b8b5df1996b2 \
--macho-segment-name NODE_SEA",
);
execSync("codesign --sign - ./dist/fauna");
execSync("chmod +x ./dist/fauna");
}

function buildSEAForWindows() {
execSync("node --experimental-sea-config .\\sea\\config.json");
fs.copyFileSync(process.execPath, ".\\dist\\fauna.exe");
// more details on signing:
// https://learn.microsoft.com/en-us/dotnet/framework/wcf/feature-details/how-to-create-temporary-certificates-for-use-during-development#installing-a-certificate-in-the-trusted-root-certification-authorities-store
// const signtool = "C:\\\"Program Files (x86)\"\\\"Microsoft SDKs\"\\ClickOnce\\SignTool\\signtool.exe";
// execSync(`${signtool} remove /s /c /u .\\dist\\fauna.exe`);
execSync(
"npx postject .\\dist\\fauna.exe NODE_SEA_BLOB .\\dist\\sea.blob ^ \
--sentinel-fuse NODE_SEA_FUSE_fce680ab2cc467b6e072b8b5df1996b2",
);
// execSync(`${signtool} sign /fd SHA256 .\\dist\\fauna.exe`);
}
7 changes: 7 additions & 0 deletions sea/config.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
{
"main": "./dist/cli.cjs",
"output": "./dist/sea.blob",
"disableExperimentalSEAWarning": true,
"useSnapshot": false,
"useCodeCache": false
}
5 changes: 4 additions & 1 deletion sea/import-meta-url.js
Original file line number Diff line number Diff line change
@@ -1 +1,4 @@
export let importMetaUrl = require('url').pathToFileURL(__filename);
// this is being used as an eslint plugin to map from CJS' __filename/__dirname
// to ESM's meta.import.url. this lets us write statements using meta.import.url
// in our source code that actually use CJS primitives after build.
export let importMetaUrl = require("url").pathToFileURL(__filename);
5 changes: 0 additions & 5 deletions sea/sea-config.json

This file was deleted.

0 comments on commit b77aab3

Please sign in to comment.