Number of simulated Clicks and behavior for validation

[neriticzone]

I am training a deepedit model from the command line. I've been able to edit the deepedit.py file to have simulated clicks on every training sample, but is there a way other than editing the source code of monailabel to increase the number of clicks (which I believe is max_interactions)?

Also unrelated, can someone explain how simulated clicks work during model validation? I just want to understand/make sure there is no leakage between the validation labels and where the clicks go. Thanks!

[diazandr3s]

Hi @neriticzone,

Thanks for opening this discussion.

The simulated clicks for DeepEdit are added randomly on the discrepancy regions. These regions are obtained from a comparison between the ground truth and prediction.

All these is done via the transform classes. This is the transform to obtain the discrepancy regions and here is the transform that randomly creates the simulated clicks.

The number of clicks are randomly generated here

The argument max_interactions doesn't define the number of clicks, but rather how many times those transforms are called.

Here is the class that interaction class that wraps this behaviour: https://github.com/Project-MONAI/MONAILabel/blob/main/monailabel/deepedit/interaction.py#L23

With regard to this question:

Also unrelated, can someone explain how simulated clicks work during model validation? I just want to understand/make sure there is no leakage between the validation labels and where the clicks go. Thanks!

I'm not sure what you mean by leakage here. But before training, the training class splits validation and training datasets here: https://github.com/Project-MONAI/MONAILabel/blob/main/monailabel/tasks/train/basic_train.py#L394

This flag defines the training or validation iteration.

For training and validation, DeepEdit alternates between interactive (click-simulation) and non-interactive iterations. The deepgrow_probability defines the probability for those types of iteration: https://github.com/Project-MONAI/MONAILabel/blob/main/monailabel/deepedit/interaction.py#L59

I hope this helps. Let us know.