Monailabel for lung nodule ct detection

[junxiant]

Hello,

I was following the tutorial for this lung nodule detection on monaibundle at https://github.com/Project-MONAI/tutorials/blob/main/monailabel/monailabel_monaibundle_3dslicer_lung_nodule_detection.ipynb

But i was wondering is there something like the radiology sample app from this link?
https://github.com/Project-MONAI/MONAILabel/blob/main/sample-apps/radiology/lib/infers/segmentation.py

The monaibundle has a script folder containing some .py scripts (trainer.py, detection_inferer.py) but im not sure how similar it is to the sample-apps. I'm asking because I am looking to do a multi-stage detection+segmentation. Is it even possible?

[diazandr3s]

Hi @junxiant,

Great question. Multi-stage systems are possible in MONAI Label. Here you can find two examples:

1/ Multistage vertebra segmentation: https://github.com/Project-MONAI/MONAILabel/tree/main/sample-apps/radiology#Multistage-Vertebra-Segmentation https://github.com/Project-MONAI/MONAILabel/blob/main/sample-apps/radiology/lib/infers/vertebra_pipeline.py

2/ Dental pipeline: Project-MONAI/MONAI#5896 https://github.com/dgmato/MONAILabel/blob/dental-pipeline/sample-apps/radiology/lib/infers/dental_pipeline.py

Those two examples are both with segmentation stages. However, there is no a limitation to create a combination of detection and segmentation.

One way could be to have both models into Radiology app format. This means, the detection bundle should be in MONAI Label Radiology format with all the classes (infer and train) and then combined them into a single pipeline.

Hope this helps,

[junxiant]

Thanks @diazandr3s ,

Since I downloaded the monaibundle versions, it is all in json format, which needs to be converted into the MONAI Label Radiology format first, and then create a pipeline which runs detection -> segmentation.

So, in the configs, infers, trainers folder i would need to have the detection + segmentation scripts, and another pipeline script in the infers folder.

I have not tried those 2 pipeline models yet, but will active learning still work? I would imagine needing to do both detection + segmentation annotation, and then submit it separately, and train the detection + segmentation models separately right?

[diazandr3s]

Hi @junxiant,

Since I downloaded the monaibundle versions, it is all in json format, which needs to be converted into the MONAI Label Radiology format first, and then create a pipeline which runs detection -> segmentation.
So, in the configs, infers, trainers folder i would need to have the detection + segmentation scripts, and another pipeline script in the infers folder.

That's right.

I have not tried those 2 pipeline models yet, but will active learning still work? I would imagine needing to do both detection + segmentation annotation, and then submit it separately, and train the detection + segmentation models separately right?

Models are trained separately.
For inference, you could consider them as a single pipeline. Here is how it works for inference: https://github.com/Project-MONAI/MONAILabel/blob/main/sample-apps/radiology/lib/infers/vertebra_pipeline.py#L143-L153

Hope this helps,

[junxiant]

Thanks, i'll look into that and try it out.

A separate question, is multi-class or multi-label possible for the fine-tuning of the lung nodule detection model? I am looking at train.json file which has the settings for the model such as num_classes. I would assume that this value would change if i were to introduce more classes.

But how should i be annotating or naming the ROI in 3D Slicer so that it is able to pick up the different classes?

[diazandr3s]

Hi @junxiant,

I think multiclasses in the detection algo is possible. My suggestion would be to start simple and add more complexity as things progress.

Kindly pinging @tangy5 to comment more on the detection algo.

[junxiant]

I was looking into the json files when the labels were submitted.

lung_006.json for two ROI with the same color seems to have the same default label naming of "ROI".
lung_045.json for two ROI with different color gave different labels. One was the default label, the other was manually added.

Seems like color matters to differentiate classes, but naming does not have any effect.

I will attach the 2 json files here too.
But I was reading the other discussion post here #1713 (comment)

So i dont think active learning would work if i add another class?

lung_006.json
lung_045.json

[junxiant]

Just an update, I dont think it works. Changing num_class in the config to 2 will just throw an error. Probably because the saved model weights and the config has different head values. Here is the error:

I tried out active learning, which was working somewhat, except for a weird error(?)
The Accuracy in 3D Slicer GUI shows 0, but on the terminal output it is showing val_coco values. Is there something i need to add in the train.json to make the value show up in 3D Slicer GUI?

[junxiant]

So to conclude, the multi-stage works.

But trying multi-class with active learning will not work. It needs to be trained from scratch.