Presentation of Test Results

[tcschmidt]

The Robot Framework produces granular XML output that allows for a detailed tracking of each step in testing: trials and results. Simple standard style-sheet processing in XML (XSLT) and HTML (CSS) grants the opportunity to produce customized presentations that help the community to quickly and deeply explore what actually happened during a test without efforts.

On the example of Hardware-in-the-Loop (HiL) testing, we tried to generate an illustrative example of this: A quick overview with the option to navigate into the details. An example of the current state can be found here. Continuous results are presented at https://ci.riot-os.org/nightlies.html.

Certainly, the quality and utility of these presentations can be improved. For this, we solicit feedback and suggestions from the community: Please provide your input in discussions of this issue or propose advancements for the style-sheets in this repo.

[kaspar030]

I think it looks very nice!

Please don't forget the scale of the target deployment. As is it shows two "test groups" (i2c, spi), for seven boards. The number of boards will be twenty times as high, and the number of tests even larger.

Maybe stacking the tests vertically would scale better?

board a
   tests periph i2c
   tests periph spi
   tests periph uart
   tests periph rng
   tests foo
   ...

board b
   tests periph i2c
   tests periph spi
   tests periph uart
   tests periph rng
   tests foo

I think whatever comes out here might also be a fit for the CI output, which is currently quite ugly plain text, but tries to present very similar data.

[MrKevinWeiss]

It would be good to identify what we would like as the first click, are we more interested in sorting by board first than test or the other way around.

I would like to propose a collapsible tab, so you only see say, all the boards. Then if you click it expands all the tests. This should be slightly more scalable (an later we could even group boards with that method).

Also, much like the current CI we have, it could be useful to jump to a first error (or first result diff) button.

@kaspar030 can the CI output standard juint or xunit xml?