Visual Explanation for Deep Metric Learning

This work explores the visual explanation for deep metric learning and its applications. We show that the proposed framework can be directly deployed to a large range of metric learning applications and provides valuable information for understanding the model. See the arxiv paper for details.

We provide runnable demos for activation decomposition and Grad-CAM (with our variants) on Image Retrieval, Person Re-identification, Face Verification, and Geo-localization. If you find this repository useful in your project, please consider citing:

@article{zhu2021visual,
  title={Visual explanation for deep metric learning},
  author={Zhu, Sijie and Yang, Taojiannan and Chen, Chen},
  journal={IEEE Transactions on Image Processing},
  year={2021},
  publisher={IEEE}
}

Requirement

- Python >= 3.5, Opencv, Numpy, Matplotlib
- PyTorch >= 1.0
- Tensorflow == 1.13.1 only for Geo-localization

Image Retrieval

Please download our pretrained model (Multi_similarity) and put it in "./Image_Retrieval/Model/". Then run the demo:

python Image_Retrieval/demo.py

You may download the CUB dataset to generate more results. Enjoy!

Person Re-identification

Please download our pretrained model (strong baseline) and put it in "./Person_Re-identification/Model/". Then run the demo:

python Person_Re-identification/demo.py

You may download the Market-1501 dataset for more results. Enjoy!

Face Verification

Please download our pretrained model (arcface) and put it in "./Face_Verification/Model/". Then run the demo:

python Face_Verification/demo.py

You may download the LFW or FIW dataset for more results. Enjoy!

Geo-localization

Please download our pretrained model package (Siamese-VGG) and put it in "./Geo-localization/Model/". Use tar -xvf model.ckpt.tar to extract. Then run the demo:

python Geo-localization/demo.py

Query pixel moving on the query street view image (left) The changing point-specific activation map on the retrieved aerial view image (right)

You may download the CVUSA dataset for more results. Enjoy! A cleaner yet stronger model is coming soon!

Reference

- https://github.com/bnu-wangxun/Deep_Metric
- https://github.com/lulujianjie/person-reid-tiny-baseline
- https://github.com/foamliu/InsightFace.git
- https://github.com/david-husx/crossview_localisation.git