Pipeline graph support for dynamic routing of frame processing

[geekscape]

Currently, the Pipelines / PipelineElements support a fixed set of StreamEvents (ERROR, STOP, OKAY) and StreamStates (ERROR, STOP, RUN). With hard-coded state management in main/pipeline.py:PipelineImpl._process_stream_event().

Also, it is intended to support user defined StreamEvents and StreamStates that are supported by a finite state machine ... main/state.py:StateMachine() ... with user defined triggers and actions.

For PipelineElements that represent proper A.I Agents ... it will be necessary to support a form of conditional and looping mechanisms, e.g SELECT, LOOP, BREAK, CONTINUE. This would enable tool (function) calling PipelineElements, including Agents to return new StreamEvents that dynamically alter the flow of frames through the graph.

Example use-cases may be ...

• Enabling a PipelineElement Router to select one or several PipelineElement Tools or Agents, which next process the frame
  • Example graph ... (Router:SELECT ToolA ToolB ToolC) # SELECT may just be syntactic sugar to make things obvious
• Enabling a PipelineElement Agent to loop until an acceptable frame output has occurred or a loop limit is reached
  • Example graph ... (Agent:LOOP ToolA:CONTINUE ToolB:BREAK) # right now ... more of an aspiration than a solid idea

Note: This discussion focuses on a static PipelineDefinition graph. A future discussion will focus on graphs, where the graph shape, e.g nodes and edge involved may change over time, i.e be more dynamic !

[pdmct]

This sounds good for modelling more dynamic behaviours.

1. Re: State machine machinery ... assuming that this will operate at the Actor ? level, what would be the mechanism for defining the 'user/custom' states / state machine etc. Will it be via some configuration for Actors or Agents, or via some extension of these classes. I think it would be good to have a unified method of creating/combining pipelines and these in the system config.
2. Re: looping, conditions. To generally implement this looping behaviour I think you only need 2 things: conditional branching and looping. You then can combine them to build whatever looping constructs you want. If we take functional calling as the main paradime then the conditional branching would be two-way conditional branching: ((branch? args) branchA branchB). and looping would be something like ((loop-while-true cond-fn args) loop-body) it exits when cond-fn is not true (this could just be a parameter value in the first instance). (args are the parameters passed into the pipeline element.) I would picture each of these as pipeline elements with branchA... and 'loop-body` as nested pipelines. These could be tarted up with some syntatic sugar to make them easier to construct but with something like these you should be able to implement any complex loop and branching structure that is needed?