Process as test (VM -> Native test)

[shivaraj-bh]

Resolves #32

Run native test

just test

TODO

• VM -> Native test
  • Postgres
  • Redis
  • Redis cluster

[srid]

Could you add a Github Actions workflow for running the new tests? As well as a justfile entry for just test.

[shivaraj-bh]

Another interesting point to consider here would be to only display the test logs, but filtering out all the other processes' logs might cause problems with visibility because if they fail we won't know why. Essentially we need a way to display logs of a given process if the process ended with a non-zero exit code.

[adrian-gierakowski]

Another interesting point to consider here would be to only display the test logs

I’m currently doing this as follows

''
  ${pc-warapper} >/dev/null 2>&1 &
  pc_pid=$!

  ${pc-warapper} process logs test -f &
  test_logs_pid=$!

  wait $pc_pid
  kill $test_logs_pid
''

On first line you could redirect to tmp file and display last X lines if exit code is non-zero.

Note that displaying only the logs of the test process will get easier once this is addressed. Logs for all processes could then be collected using log_location at the top level of the config file.

[shivaraj-bh]

On first line you could redirect to tmp file and display last X lines if exit code is non-zero.

Thanks! Will give it a shot.

[shivaraj-bh]

CI works 🥳

[srid]

On macOS (Nix 2.17), I get:

❯ just test
nix flake check test/ --override-input services-flake . -L
warning: not writing modified lock file of flake 'git+file:///Users/srid/code/services-flake?dir=test':
• Updated input 'services-flake':
    'github:juspay/services-flake/f171932d130c3ce5184b69144012a225b2e28868' (2023-06-19)
  → 'git+file:///Users/srid/code/services-flake?ref=refs/heads/process-as-test&rev=9928f03781cba69263b83239c6c7bd26d6a1ef97' (2023-08-10)
error: builder for '/nix/store/c53s53w0xis4ak8jaw1df8cninh92d47-redis-test.drv' failed with exit code 1
error: build of '/nix/store/c53s53w0xis4ak8jaw1df8cninh92d47-redis-test.drv', '/nix/store/d9ssfys8nawjym0cgm26g6m5czamhzpa-redis-cluster-test.drv' failed
error: Recipe `test` failed on line 14 with exit code 1

This is odd,

❯ nix log /nix/store/c53s53w0xis4ak8jaw1df8cninh92d47-redis-test.drv
warning: The interpretation of store paths arguments ending in `.drv` recently changed. If this command is now failing try again with '/nix/store/c53s53w0xis4ak8jaw1df8cninh92d47-redis-test.drv^*'

Tests run on NixOS though.

[shivaraj-bh]

On macOS (Nix 2.17), I get:

❯ just test
nix flake check test/ --override-input services-flake . -L
warning: not writing modified lock file of flake 'git+file:///Users/srid/code/services-flake?dir=test':
• Updated input 'services-flake':
    'github:juspay/services-flake/f171932d130c3ce5184b69144012a225b2e28868' (2023-06-19)
  → 'git+file:///Users/srid/code/services-flake?ref=refs/heads/process-as-test&rev=9928f03781cba69263b83239c6c7bd26d6a1ef97' (2023-08-10)
error: builder for '/nix/store/c53s53w0xis4ak8jaw1df8cninh92d47-redis-test.drv' failed with exit code 1
error: build of '/nix/store/c53s53w0xis4ak8jaw1df8cninh92d47-redis-test.drv', '/nix/store/d9ssfys8nawjym0cgm26g6m5czamhzpa-redis-cluster-test.drv' failed
error: Recipe `test` failed on line 14 with exit code 1

This is odd,

❯ nix log /nix/store/c53s53w0xis4ak8jaw1df8cninh92d47-redis-test.drv
warning: The interpretation of store paths arguments ending in `.drv` recently changed. If this command is now failing try again with '/nix/store/c53s53w0xis4ak8jaw1df8cninh92d47-redis-test.drv^*'

Tests run on NixOS though.

This also happens on my macOS with Nix 2.13.3. Additionally I also tried running the example flake with just ex after updating nixpkgs in example/flake.lock to use the latest PC and the example app doesn't run at all.

I tried on different macOS (with Nix 2.15.1) where PC was never ran before and it seems to work fine there. This is weird, have to debug this.

By MacOS I mean aarch64-darwin in both the devices mentioned above.

[shivaraj-bh]

@srid just test is working on mac, the error wasn't conclusive but it looks like having each PC package run on different swagger endpoint port makes it work.

A nicer implementation will be to disable swagger endpoint so that we won't have this conflict.

[srid]

A nicer implementation will be to disable swagger endpoint so that we won't have this conflict.

Yea, I was thinking of an upstream contribution to that end. Or we can set port to 0 by default (in process-compose-flake), forcing the user to set it explicitly if they want it.

just test runs fine for me now.

[srid]

Display test logs if non-zero exit code

Can we do this separately in a different PR? I feel like we won't even need it, but let's see if a need actually arises over time.

[srid]

LGTM once process-compose-flake is updated.

[shivaraj-bh]

@srid Looks like the CI is running forever when I set disabled = true; for test process

[shivaraj-bh]

Looks like we can only override keys that are not already set in the base yaml configuration. We can either upstream the change in process-compose or use two overlays (one for package that disables test and another for testPackage that enables it and adds exit_on_end).

[srid]

Oh, I thought we are automatically disabling the test process. I think we should upstream a change to process-compose-flake such that outputs.package too uses an override config that disables the test process if there is one. The user doesn't have to set disabled = true manually on their test process.

After all, "test" has a special name in process-compose-flake, and it should behave uniformly across all outputs it produces.

[srid]

Let's get this merged, after switching upstream flake url.

[shivaraj-bh]

Done!

[srid]

Great, you can (squash) merge it yourself.