-
Notifications
You must be signed in to change notification settings - Fork 5
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Implement brioche test
subcommand
#94
Comments
I think mirroring a common testing library would work pretty well for Brioche: import * as std from "std";
import * from "test";
export const project = {
name: "curl",
version: "1.23.45",
};
export default function curl() { /* ... */ }
describe("curl", () => {
it("has the right version", async () => {
const expectedVersion = project.version;
const actualVersion = await std.runBash`curl --version > "$BRIOCHE_OUTPUT"`.dependencies(curl()).toFile().read();
expect(expectedVersion).toEqual(actualVersion);
});
}); In this case, I went with a syntax like Ruby RSpec (that's at least where I first saw this style of test, and it's been copied for testing frameworks like Jest and Vitest in the JS ecosystem) We could also go with a simpler model, more like what Rust does: import * as std from "std";
std.test("test curl version", async () => {
const expectedVersion = project.version;
const actualVersion = await std.runBash`curl --version > "$BRIOCHE_OUTPUT"`.dependencies(curl()).toFile().read();
std.assert(expectedVersion == actualVersion);
}); (and of course, either choice could have all sorts of utilities for more complex assertions or helpers for getting the output from a Bash command) The key is that Then, the runtime would either run or skip the test, depending on if the project is being called with |
Actually, I'm pretty split here. Using a simple At least as an initial implementation, it seems like just using a |
Currently, the only way to validate if a Brioche package is working correctly is to try building and running it. This has a few problems:
std
package and packages that just wrap a library don't have an "obvious" way to get run... at least not one that would be usable via thebrioche run
command line--version
,version
,--help
, orhelp
would work, but that's not guaranteed, and there's no way to know whichbrioche run
runs on the host machine. This means its possible for a package to work withbrioche run
but to fail when used within Brioche, e.g. by calling it withstd.runBash
As a workaround, a few packages (namely,
openssl
, plusbat
in #70) expose an export namedtest
. This can be used explicitly by runningbrioche build -e test
, which allows for writing arbitrary assertions that can run in the Brioche sandbox natively. But, just using atest
export has its own downsides:test
is just a normal export, it must return a recipe that gets baked. Plus, that recipe must also write to$BRIOCHE_OUTPUT
explicitly, otherwise the build will fail.A solution to both problems is to add a new
brioche test
command, specifically designed for writing tests. We also don't need to be bound by the same rules that exports follow today.To start with, we should focus on writing tests for packages, as this is the clearest need. At some point, it could make sense to expand the scope and offer features that help users write tests more generally for their own projects
The text was updated successfully, but these errors were encountered: