diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 69f42cd9..d50f075b 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -73,12 +73,13 @@ For testing of **binary image outputs** we recommend using the `mae` metric (mea ```xml ``` +The macro also ensures that the image contains two distinct label values, which are not interchangable. For testing of non-binary **label map outputs** with interchangeable labels, we recommend using the `iou` metric (one minus the *intersection over the union*). With the default value of `eps` of 0.01, this asserts that there is no labeled image region with an *intersection over the union* of less than 99%: ```xml ``` -At the moment it is not possible to *pin* specific labels, for example to verify that the background is assigned the correct label, but this will hopefully be added in the future. +Label 0 is commonly connotated as the image background, and is not interchangable by default. Use `pin_labels=""` to make it interchangable. For testing of **intensity image outputs** we recommend the `rms` metric (root mean square), because it is very sensitive to large pixel value differences, but tolerates smaller differences: ```xml diff --git a/macros/tests.xml b/macros/tests.xml index e0853472..a09c7d4e 100644 --- a/macros/tests.xml +++ b/macros/tests.xml @@ -8,7 +8,7 @@ token_metric="mae" token_eps="0.01"> - + @@ -19,11 +19,12 @@ + token_eps="0.01" + token_pin_labels="0"> - + @@ -53,7 +54,7 @@ token_metric="mae" token_eps="0.01"> - + @@ -66,9 +67,10 @@ name="tests/label_image_diff/element" tokens="name,value,ftype,metric,eps" token_metric="iou" - token_eps="0.01"> + token_eps="0.01" + token_pin_labels="0"> - +