README

WHY 9PM
-------
9pm is designed to address the complexity of interactively managing different
systems simultaneously.

Consider this setup
 ___________    _______    _____
|           |  |       |  |     |
| localhost |--| encom |--| clu |
|___________|  |_______|  |_____|
                   |
                ___|___
               |       |
               | flynn |
               |_______|

You're running on localhost and you want to start a server on clu and connect
to it from flynn. Then you want to make sure that the client (on flynn) succeeds
and that the hostname of flynn are shown in in the server output on clu.

This is how you could do that with 9pm.

shell "server"
ssh "encom"
ssh "clu"
start "./server"

shell "client"
ssh "encom"
ssh "flynn"
set hostname [execute "hostname" 0]
execute "./client" 0

shell "server"
expect {
	"$hostname connected" {
		result OK "$hostname connected to server"
	}
}
set rc [finish]
output INFO "Server returned $rc"

CODING STYLE
------------
o Spaces not tabs
o A line should not be longer than 100 chars
o {} embracing are not padded (if {foo} not if { foo })
o No one-line if statements or procedures
o Bool return values are "FALSE" or "TRUE"

NAMESPACE AND SCOPE
-------------------
o Top level namespace (::9pm) should be empty ("fatal" is a current exception).
o Second level namespace names should be unique i.e. not collide with
  anything in TCL/Expect/tcllib.
o Users should be able to use "namespace path ::9pm" to recursively import all
  second level namespaces and there childes.
o Procedures should strive to use variables inside there own namespace.
o All internal references outside of own namespace should be absolute from
  global (::9pm::foo::bar).

RETURN VALUES
-------------
Defined bad return values (1-10 is reserved by the framework)
2 - Fatal wrapper trigger

EXECUTION
---------
When executing a 9pm script without the harness you need to tell tcl where
this library is placed. This can be done via the TCLLIBPATH environment
variable. Something like "export TCLLIBPATH='/usr/share/tcltk/ ~/code/9pm'"

HARNESS
-------
9pm.py is a harness and test suite manager for 9pm testcases.
The tool can optionally record the result of the run to a JUnit XML file that
can be passed on for further analyses.

* Basic suites

  Suites are YAML files pointing out testcases and/or other testsuites,
  example:

     $ cat suite.yaml
     - case: cases/foo.tcl
     - case: cases/bar.tcl

  Here we see a suite that contains two testcases foo.tcl and bar.tcl.
  These two testcases will be executed om sequence as they appear in the
  suite description. It is also possible to include other suites in
  suites, example:

     $ cat nestedsuite.yaml
     - case: cases/foo.tcl
     - case: cases/bar.tcl
     - case: subsuite.yaml

     $ cat subsuite.yaml
     - case: cases/foo.tcl
     - case: cases/baz.tcl

   Here nestedsuite.yaml includes the two testcases foo.tcl and bar.tcl,
   it also includes the testsuite subsuite.yaml which in turn includes
   the testcase foo.tcl (again!) and baz.tcl. A path is deemed to be a
   testsuite if the file suffix is .yaml, all other suffixes are treated
   as testcases.

* Suite namespace

  In order to keep track of who included what testcase and to generate
  traceable output 9pm.py runs each testcase and testsuite in a
  namespace, if we examine the namespace for suite.yaml from above
  it looks like:

     suite/foo
     suite/bar

  This is a sane execution namespace without collisions. If we examine
  the namespace of nestedsuite.yaml that pulls in foo.tcl twice (once in
  nestedsuite.yaml and once in subsuite.yaml) it looks like:

     nestedsuite/foo
     nestedsuite/bar
     nestedsuite/subsuite/foo
     nestedsuite/subsuite/baz

  This is also a sane execution namespace without collisions since
  testcase foo.tcl is included from different suites 9pm.py can tell
  the two invocations apart. But what if I wish to include the same
  testcase multiple times in the same testsuite you ask? No problem,
  9pm.py looks for name hints in the testsuite description, example:

    $ cat namedsuite.yaml
    - case: cases/foo.tcl
    - case: cases/foo.tcl
      name: foo1
    - case: cases/foo.tcl
      name: foo2

  Testsuite namedsuite.yaml would result in the execution namesapace:

    namedsuite/foo
    namedsuite/foo1
    namedsuite/foo2

  In this testsuite testcase foo.tcl is executed three times in a row
  but under different names so no namespace collisions occur.

  Naturally name hints works the same for testsuites!

* Pass options through 9pm to testcases

  Custom options can be passed to testcases. Either on the command line
  when starting 9pm or through suites.

  9pm.py -o "opt1" -o "opt2"

  $ cat debugsinglesuite.yaml
  - case: cases/foo.tcl
  - case: cases/bar.tcl
    opts:
      - "suite-supplied-option1"
      - "suite-supplied-option2"

  The opts parameter allows you to pass a YAML list of options to any
  testcase invocation. This also works for testsuites (all cases in a
  suite receive the same options:

  $ cat debugeth0network.yaml
  - case: cases/foo.tcl
  - suite: network.yaml
    opts:
      - "-d"
      - "eth0"
  - case: cases/bar.tcl

  This will pass the options '-d' and 'eth0' to all testcases included
  from the network.yaml testsuite.

* Run script after test case failure (onfail)
  $ cat suite.yaml
  - case: cases/foo.tcl
  - case: cases/bar.tcl
    onfail: "cleanup.sh"

  This will run the script "cleanup.sh" if the test case "bar.tcl" fails. The
  path of "cleanup.sh" is relative to that of "bar.tcl".