Execution error that may be due to missing tty or streams for Multiple layers of Dax+Jupyter Notebooks

[zph]

👋 Thank you for dax! I've been building a toolkit for dynamic runbooks that heavily uses it and I'm very happy with the interfaces and behavior.

I found a case where there's an unexpected behavior when using dax library in Deno's Jupyter notebooks.

I'll describe it here and please let me know if you want me to post it instead as a deno jupyter issue on main repo.

Error

When I create a e2e testing harness in Deno (v2.1 and others) that uses dax (v0.42.0 and others) that calls a python cli tool (zph/runbook) to non-interactively execute a deno jupyter notebook that contains dax executions of simple commands (git log), I get this error:

❯ deno test -A tests/cli_test.ts
Check file:///Users/zph/src/runbook/main/tests/cli_test.ts
running 1 test from ./tests/cli_test.ts
run ...
------- output -------
/var/folders/3j/2s7slhwx7n3f0hm6m3q0_2r40000gn/T/50e926c6e6957e68/output.ipynb
----- output end -----
run ... FAILED (1s)

 ERRORS 

run => ./tests/cli_test.ts:134:11
error: AssertionError: Values are not equal.


    [Diff] Actual / Expected


-   \rExecuting:   0%|          | 0/15 [00:00<?, ?cell/s]Warning "deno jupyter" is unstable and might change in the future.\n
-   \rExecuting:   7%|▋         | 1/15 [00:00<00:10,  1.35cell/s]\rExecuting:  27%|██▋       | 4/15 [00:00<00:01,  5.84cell/s]\rExecuting:  33%|███▎      | 5/15 [00:01<00:02,  4.99cell/s]\n
-   Traceback (most recent call last):\n
-     File "/Users/zph/src/runbook/main/.venv/bin/runbook", line 10, in <module>\n
-       sys.exit(cli())\n
-                ~~~^^\n
-     File "/Users/zph/src/runbook/main/.venv/lib/python3.13/site-packages/click/core.py", line 1161, in __call__\n
-       return self.main(*args, **kwargs)\n
-              ~~~~~~~~~^^^^^^^^^^^^^^^^^\n
-     File "/Users/zph/src/runbook/main/.venv/lib/python3.13/site-packages/click/core.py", line 1082, in main\n
-       rv = self.invoke(ctx)\n
-     File "/Users/zph/src/runbook/main/.venv/lib/python3.13/site-packages/click/core.py", line 1697, in invoke\n
-       return _process_result(sub_ctx.command.invoke(sub_ctx))\n
-                              ~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^\n
-     File "/Users/zph/src/runbook/main/.venv/lib/python3.13/site-packages/click/core.py", line 1443, in invoke\n
-       return ctx.invoke(self.callback, **ctx.params)\n
-              ~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n
-     File "/Users/zph/src/runbook/main/.venv/lib/python3.13/site-packages/click/core.py", line 788, in invoke\n
-       return __callback(*args, **kwargs)\n
-     File "/Users/zph/src/runbook/main/.venv/lib/python3.13/site-packages/click/decorators.py", line 33, in new_func\n
-       return f(get_current_context(), *args, **kwargs)\n
-     File "/Users/zph/src/runbook/main/runbook/cli/commands/run.py", line 39, in run\n
-       pm.execute_notebook(\n
-       ~~~~~~~~~~~~~~~~~~~^\n
-           input_path=filename,\n
-           ^^^^^^^^^^^^^^^^^^^^\n
-           output_path=output,\n
-           ^^^^^^^^^^^^^^^^^^^\n
-       )\n
-       ^\n
-     File "/Users/zph/src/runbook/main/.venv/lib/python3.13/site-packages/papermill/execute.py", line 131, in execute_notebook\n
-       raise_for_execution_errors(nb, output_path)\n
-       ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^\n
-     File "/Users/zph/src/runbook/main/.venv/lib/python3.13/site-packages/papermill/execute.py", line 234, in raise_for_execution_errors\n
-       raise error\n
-   papermill.exceptions.PapermillExecutionError: \n
-   ---------------------------------------------------------------------------\n
-   Exception encountered at "In [3]":\n
-   Stack trace:\n
-   Error: Exited with code: 128\n
-       at CommandChild.pipedStdoutBuffer (https://jsr.io/@david/dax/0.42.0/src/command.ts:785:19)\n
-       at eventLoopTick (ext:core/01_core.js:175:7)\n
-   \n

With the final portion being most relevant and the rest for context of the many layers going on.

-   Exception encountered at "In [3]":\n
-   Stack trace:\n
-   Error: Exited with code: 128\n
-       at CommandChild.pipedStdoutBuffer (https://jsr.io/@david/dax/0.42.0/src/command.ts:785:19)\n
-       at eventLoopTick (ext:core/01_core.js:175:7)\n

The test definition is: https://github.com/zph/runbook/blob/main/tests/cli_test.ts#L134-L140

Deno.test("run", async (t) => {
  const {runbook, dir} = await setup();
  const cmd = await runbook(["run", "--no-interactive", "runbooks/binder/_template-deno.ipynb", "--output", "output.ipynb"]);
  assertSnapshot(t, { stdout: cmd.stdout, stderr: cmd.stderr, exitCode: cmd.code });
  const output = await Deno.readTextFile([dir, "output.ipynb"].join("/"));
  assertSnapshot(t, output);
});

And in essence it's calling an underlying shell command as follows:

runbook  run --no-interactive runbooks/binder/_template-deno.ipynb --output output.ipynb

Running that command by itself without the deno/dax test harness succeeds with exit code of 0.

If I replace the cell's call to Dax with Deno's Command interface, it succeeds when run through the otherwise failing test harness (eg):

const gitLog = new Deno.Command("git", {
  args: ["log"],
});

try {
  const output = await gitLog.output();
  console.log(new TextDecoder().decode(output.stdout));
} catch (error) {
  console.error("Error running git log:", error);
}

Observations

• It seems to happen when I have recursive use of dax (dax-in-dax) but not in all circumstances.
• I've only found it to occur when running deno through the deno jupyter kernel
• It's happening reproducibly for me in this case of non-interactive jupyter runs where I'm executing the run itself also from deno.
• I'm wondering if there's a problem with stdout/stderr not behaving like normal inside the non-interactive jupyter environment for dax, when dax is both the outer caller (test harness) and the inner caller (a jupyter cell).
• Deno's builtin Command interface doesn't demonstrate this issue

What I've tried

1. Different versions of dax
2. Different configuration values for stdout/stderr in build$ among "piped", etc including custom streams/byte arrays
3. Different versions of deno (v2.1 and v1.45)
4. Replacing dax with the Deno Command interface (works but that lacks the conveniences of dax)

Next Steps

I'm not sure where to investigate next. I'm happy to try branches/patches to see if they resolve the issue or to provide additional debugging or try different configurations of dax to narrow this down.

Thanks!