Nightingale Detecting Active Tuberculosis Bacilli - 2024

This repository contains the winning solution for the Detecting Active Tuberculosis Bacilli - 2024 Contest hosted by Nightingale Open Science and Wellgen Medical.

Zsolt Bedőházi, András M. Biricz

How our approach works?

Our proposed methodology for TB detection utilizes a weakly-supervised framework, effectively addressing data privacy and accessibility challenges by eliminating the dependency on external datasets and annotations made by experts. Employing transfer learning, we leverage state-of-the-art pre-trained vision encoders to extract relevant features from TB images in the absence of direct annotations. Subsequently, these features are utilized to train various self-supervised models (Transformers and MIL), enabling them to learn directly from the data using only image-level labels. Our approach provides a rapid, scalable, and efficient solution for TB detection from sputum microscopy images, improving diagnostic capabilities in resource-limited areas.

Usage

Extract patches from images

01_raw_patches/
  └──extract_patches_and_coords.py

Generate patch embeddings

02_patch_embeddings/
  ├── generate_dinov2-vitlarge_embeddings.py
  ├── generate_dinov2-vitsmall_embeddings.py
  └── generate_uni_embeddings.py

Training

03_training/
  └──hipt_stage3_on_embeddings_bag/
      └── hipt_stage3_training-balanced-folds-cross-val-uni-efficientloader-scheduler.ipynb
  └──transformer_on_embeddings_bag/
      ├── train_transformer_cls_on_embeddings_bag.py
      └── train_transformer_cls_on_embeddings_bag_multi_branch.py

Prediction

04_prediction/
  └──hipt_stage3_on_embeddings_bag/
      └── predict_tb_with_hipt_stage3_dinov2-vit_10fold_improved.ipynb
  └──transformer_on_embeddings_bag/
      ├── predict_tb_with_transformer_dinov2-vit_10fold_improved.ipynb
      └── predict_tb_with_transformer_dinov2-vit-multibranch_10fold_improved.ipynb
  └──predict_tb_fusion.ipynb

Pre-trained models

Model	Arch	# of params	# of train images	Download
dino2-small	ViT-S/14 distilled	21 M	142 M	link
dinov2-large	ViT-L/14 distilled	300 M	142 M	link
UNI	ViT-L/14 distilled	300 M	100 M	link

License

This project is licensed under the Apache License 2.0 - see the LICENSE file for details.