Idea: train a multitomodal model to predict patients ouctome

[valosekj]

Just for the record, I'm documenting here an idea from @plbenveniste raised during the 2024-11-07 NeuroPoly lab meeting.

Idea: instead of having a two-model approach (model 1: lesion and cord segmentation and morphometrics calculation, model 2: patients' outcome predictive model based on MRI morphometrics and clinical scores), we could train a single multimodal model (trained simultaneously on MRI scans and text clinical data).

TODO:

• add refs to some recent foundation multimodal models
• add ref to the paper from Toronto (in DCM?)