Use active learning based on uncertainty to augment dataset

[jcohenadad]

This PR introduces the following algorithm:

• Run predictions using models published in https://github.com/ivadomed/model_seg_mouse-sc_wm-gm_t1/releases/tag/v0.2
• Compute STD across predictions
• Average STD per slice (ignore zeros and nans)
• If STD is above a threshold, remove prediction for that slice.
• Save predictions under derivatives/labels_active_learning
• Train new model with uncertainty-filtered predicted labels

Fixes #31

[jcohenadad]

Output prediction_std on sub-mouse1/anat/sub-mouse1_chunk-1_T1w with version 534f16a

prediction_std.csv

Based on this plot, slices 416 --> 440 have the highest STD. Let's look at prediction STD from these slices:

Yup! that looks about right:

[jcohenadad]

Few things to consider:

• Would an STD threshold be generalizable across the whole dataset?
• How reliable is STD computed on very few pixels?

[jcohenadad]

How reliable is STD computed on very few pixels?

In this example, STD is 0.000305 (relatively low):

But there is no pixel picked up by majority voting!

In this other example, STD is high (0.03803):

But the segmentation is pretty good!

So, I need to find another way to filter based on STD (or another metric...)

[jcohenadad]

Few ideas to address #35 (comment)

• A "good looking" segmentation has high uncertainty only at its edges. So maybe I could ignore STD computed at the edge of the segmentation?
• Instead of computing STD, compute the total overlap (a bit like a Dice, but across more than 2 labels).