fix(api): New protocols print a JSON "run log" from opentrons_execute and opentrons.execute.execute()

[SyntaxColoring]

Overview

Closes RSS-283.

When running PAPIv≥2.14 or JSONv≥6 protocols, opentrons_execute will now output a "run log" to stdout like this:

{"id": "056a8b70-0f46-43ca-b0eb-847671d31eff", "createdAt": "2023-09-27T18:33:48.154347+00:00", "commandType": "custom", "key": "3ca2f4eb308dc93ebab03777f50b0d85", "status": "running", "params": {"legacyCommandType": "command.COMMENT", "legacyCommandText": "Hello, world!"}, "result": null, "error": null, "startedAt": "2023-09-27T18:33:48.155772+00:00", "completedAt": null, "intent": null}
{"id": "f4d94b1d-6278-45ca-a525-8f811bbdcaa4", "createdAt": "2023-09-27T18:33:48.166984+00:00", "commandType": "loadPipette", "key": "72836212bea2e508afa3f07b3c296c6c", "status": "running", "params": {"pipetteName": "p20_multi_gen2", "mount": "left", "pipetteId": null}, "result": null, "error": null, "startedAt": "2023-09-27T18:33:48.168365+00:00", "completedAt": null, "intent": null}

And the same objects are exposed by opentrons.execute.execute()'s emit_runlog callback.

Test Plan

• Test this on a Flex by running python3 -m opentrons.execute path_to_some_protocol.py with a protocol that specifies "apiLevel": "2.15".
• Test this on an OT-2 by running opentrons_execute path_to_some_protocol.py with a protocol that specifies "apiLevel": "2.14" or above.

Changelog

• Add a new helper to allow monitoring a ProtocolEngine for ongoing commands.

  This works by subscribing to every state update and seeing if the currently running command has changed since the last one. If it has, we emit a "command running" or "command no longer running" event. Conceptually, that event structure matches the "before" and "after" messages that the Python Protocol API has long published.

• Wire that helper up to the opentrons_execute CLI and the opentrons.execute.execute() function.

  opentrons.execute.execute() has a quirky interface that we need to preserve for now, so this takes some shoehorning. For this ticket, I'm not concerned about how the protocol's activity gets printed—just that it gets printed somehow. So I'm doing that by reporting each command as a "comment" where the message is the command's JSON text. There's obviously a lot of room for improvement here. RSS-320 tracks making these messages more nicely human-readable. And if there's a use case for it, we could also go the other way, officially exposing the command in a machine-readable way.

Review requests

• Is this compatible with existing public-facing interfaces, in the sense that it won't outright break any code?
• See my inline comments.

Risk assessment

Low. This is tricky to test, but it's also fairly nonintrusive.

[codecov]

Codecov Report

Merging #13629 (1a8c1f7) into chore_release-7.0.1 (727030b) will decrease coverage by 0.01%.
The diff coverage is n/a.

Additional details and impacted files

@@                   Coverage Diff                   @@
##           chore_release-7.0.1   #13629      +/-   ##
=======================================================
- Coverage                71.29%   71.29%   -0.01%     
=======================================================
  Files                     2427     2427              
  Lines                    68229    68226       -3     
  Branches                  7968     7968              
=======================================================
- Hits                     48647    48644       -3     
  Misses                   17715    17715              
  Partials                  1867     1867              

Flag	Coverage Δ
app	68.90% <ø> (ø)
g-code-testing	96.44% <ø> (ø)
notify-server	89.13% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

Files	Coverage Δ
api/src/opentrons/execute.py	54.09% <ø> (-0.75%)	⬇️
...i/src/opentrons/protocol_engine/protocol_engine.py	100.00% <ø> (ø)
...opentrons/protocol_engine/state/change_notifier.py	100.00% <ø> (ø)
...pi/src/opentrons/protocol_engine/state/commands.py	99.45% <ø> (-0.01%)	⬇️
api/src/opentrons/protocol_engine/state/state.py	100.00% <ø> (ø)
...opentrons/protocol_runner/legacy_context_plugin.py	100.00% <ø> (ø)

[SyntaxColoring]

Things I do not like about this:

• All of the footguns described in the docstring.
• It's callback-based, which can be spaghetti.

Things I like about this:

• Making the mechanism this generic—as opposed to something more specific like wait_for_commands()—lets us build more logic outside of ProtocolEngine. This is really nice for curbing ProtocolEngine complexity.
• This is is roughly compatible with my experiments in notifications for the HTTP API. I'm expecting various internal classes like ProtocolEngine and AnalysisStore to grow interfaces that let callers monitor them for changes in a coarse-grained way. Then, higher-level endpoints filter those notifications down and return only what the client actually cares about.
• Despite callbacks being ugly, they are suited to the actual requirements of RSS-238. The ultimate user-facing interface is callback-based. If we want async generators later, it's easier to build those atop callbacks than to do it the other way around.

[jbleon95]

Nice, I think this is the most straightforward but sensible way to add this feature to PE-run CLI.

[TamarZanzouri]

this is new to me :-)

[SyntaxColoring]

Yeah...Python. 😬

[TamarZanzouri]

I agree with Jeremy. I like this approach. clean and straightforward. nice work!