Add checkPhase to ensure compiled packages actually work

[considerate]

Missing FFI files is not an error so to ensure a package set actually works together running tests is important.

[thomashoneyman]

Unfortunately the registry does not include test directories by default (because build tools like Spago and Pulp ignore them) but we could reconsider that if you want to raise an issue.

[considerate]

I’m not sure to what extent this should be enforced. The issue with tests is that some tests may be flaky or depend on having an Internet connection. There’s also the issue of how to treat packages with very extensive and computationally heavy tests.

Nonetheless, when trying to establish package sets for purenix I ran into the issue that it was difficult to verify if the built packages actually:

1. Compiled the sources into meaningful nix files
2. Created outputs that would be able to construct a cohesive application or library
3. Had an implementation that wouldn’t just result in a lot of runtime errors

I think some of the above could be done by inspecting the purs logs for warnings and treating those as failures but that only gets you so far. Hence, I want to move forwards with optionally running tests for each package in a package set.

[considerate]

@thomashoneyman Your point completely escaped me. It would indeed be nice if the test directories were included so that it’s possible to test the package sets even though spago won’t do so by default.

It wouldn’t be nice to fetch the test directories separately since then it wouldn’t give as much confidence that the published (tarball) version works.

[considerate]

Apart from including the test sources in the registry storage backend the registry would also need to support declaring test dependencies.

The purs.json file format doesn't have a field for test dependencies like spago does.