Simulator

How the Sossity Simulator works

See the [https://github.com/22Acacia/sossity](Sossity README) for command line args, and the Test Config file format for configuration options.

The Sossity simulator creates a virtual series of pipelines, processes, and flows. It uses the identical planning code from the Sossity planner, so all data output should be identical to production code.

Simply create input files as proper JSON objects in a text file ([{obj1},{obj2},{obj3}] etc) and the simulator takes care of everything else.

Simulator Internals

The simulator relies on the homoiconic nature of Clojure to make it easy to build an execute a graph of processes.

1. The config files are parsed with sossity.core and the resource dependency graph is generated
2. The simulator does a breadth-first traversal of this graph and annotates the data structure with Processes and Channels. These represent Pubsubs and Cloud Dataflow jobs in production.
3. In the nodes of the graph, the go-loop from core.async in every process loads the pipeline jars and execution class form the config file into a JVM thread
4. pubs and subs from core.async channels are created on the edges of the graph.
5. At every Source, a process loads the .json file indicated in the config file and writes the data to a pub.
6. Data is passed through channels and processes, transformed until it is output to the filesystem in the test-output directory for every Sink.