SMILE

SMILE: Self-Motivated Multi-instance Learning for Whole Slide Image Classification (Under Review)

Tingting Zheng, Kui jiang, Hongxun Yao

Harbin Institute of Technology

News

• [2024/10/24] The slide contain the magnification and other related metadata properties, which can be accessed through slide.properties using the openslide library datasets/get_WSI_level.py.
• [2024/10/23] We provide the training and testing code, along with the pre-trained weights on the BRCAS dataset, which were pre-trained with a 512-dimensional ResNet18 on ImageNet. By using the preprocessed features dataset_hsfiles/bracs_re18_imag_10xtest and pre-trained model save_modes/bracs_re18_imag_10x, our results can be reproduced. We will also further optimize the code and provide the complete pre-trained weights.
• [2024/07/30] We provide pre-processed data methods and features.
• [2024/03/30] The repo is created.

Pre-requisites:

• Linux (Tested on Ubuntu 18.04)
• NVIDIA GPU (Tested on 3090)

Dependencies:

Package                       Version
----------------------------- ------------
faiss-gpu               1.7.2
h5py                    3.9.0
numpy                   1.24.3
nystrom-attention       0.0.11
openslide-python        1.2.0
pandas                  1.1.4
pip                     23.1.2
python-dateutil         2.8.2
scikit-learn            1.3.0
scipy                   1.11.1
sklearn-pandas          2.2.0
tensorboard             2.13.0
thop                    0.1.1
torch                   2.0.0+cu117
torchaudio              2.0.1+cu117
torchmetrics            1.2.0
torchvision             0.15.1+cu117
tqdm                    4.65.0

Usage

Dataset

Preprocess TCGA Dataset

We use the same configuration of data preprocessing as DSMIL.

Preprocess CAMELYON16, ESCA and BRACS Datasets

We use CLAM to preprocess slides. For your own dataset, you can modify and run create_patches_fp_Lung.py and extract_features_fp_LungRes18Imag.py.

The data used for training, validation and testing are expected to be organized as follows:

DATA_ROOT_DIR/
    ├──DATASET_1_DATA_DIR/
        └── pt_files
                ├── slide_1.pt
                ├── slide_2.pt
                └── ...
        └── h5_files
                ├── slide_1.h5
                ├── slide_2.h5
                └── ...
    ├──DATASET_2_DATA_DIR/
        └── pt_files
                ├── slide_a.pt
                ├── slide_b.pt
                └── ...
        └── h5_files
                ├── slide_a.h5
                ├── slide_b.h5
                └── ...
    └── ...

Dataset Preparation

We provide a part of the extracted features to reimplement our results.

Camelyon16 Dataset (20× magnification )

Model	Download Link
ImageNet ResNet50 Testing	Download
ImageNet ResNet50 Training and validation	Download

The preprocessed features, as well as the training, validation, and testing splits, are all derived from MMIL-Transformer.

TCGA Dataset (20× magnification)

Model	Download Link
SimCLR ResNet18	Download

The preprocessed features, as well as the training, validation, and testing splits, are all derived from MMIL-Transformer.

Bracs Dataset (10× magnification)

Model	Download Link
ImageNet supervised ResNet18	Download
SSL ViT-S/16	Download

The preprocessed features, as well as the training, validation, and testing splits, are all derived from ACMIL.

We appreciate their contributions and outstanding work for the entire community.

Acknowledgments

Our SMILE mainly borrows from CLAM, DTFD-MIL, TransMIL, MMIL-Transformer, IBMIL and PAMIL. Thanks for these excellent open-source works!

Contact

If you have any questions or suggestions, feel free to contact me. Email: [email protected]; [email protected]

Cite this work

@inproceedings{zheng2024dynamic,
    title={SMILE: Self-Motivated Multi-instance Learning for Whole Slide Image Classification},
    author={Zheng, Tingting and
            Jiang, Kui and
            Yao, Hongxun},
    year={2024}
}