scripts

Scripts for exploring foundation models based VPR

Table of contents

• Scripts for exploring foundation models based VPR
  • Table of contents
  • Contents
    • DINOv2
    • DINO
    • CLIP
    • CosPlace
    • Dataset Cluster Visualization
    • Bash Scripts

Contents

Scripts that perform methods and tasks

DINOv2

S. No.	File	Description
1	dino_v2_vlad.py	VLAD over DINOv2 ViT features
2	dino_v2_gem.py	GeM pooling over DINOv2 ViT features
3	dino_v2_global_vpr.py	Use the CLS token of Dino-v2 as the global descriptor
4	dino_v2_vlad_viz_layers.py	Visualize the DINOv2 Vocabulary for different ViT layers
5	dino_v2_sim_facets.py	Similarity maps between/across ViT facets (in a layer) for query pixel (in a target image) using Dino-v2 features
6	dino_v2_global_vocab_vlad.py	Generate VLAD cluster centers using collection of datasets (for domain-specific map/vocabulary)
7	dino_v2_gp.py	Pooling (max or average) DINOv2 ViT features
8	dino_v2_vlad_global_vocab.py	IGNORE: VLAD over DINOv2 ViT features using a global vocabulary (concatenated all datasets). A special case of the dino_v2_global_vocab_vlad.py script.

DINO

S. No.	File	Description
1	dino_vlad.py	VLAD over DINO ViT extracted features
2	dino_gem.py	GeM over DINO ViT extracted features
3	dino_global_vpr.py	VPR using CLS token
4	dino_global_vocab_vlad.py	VLAD over DINO ViT extracted features using custom (global map & domain-specific) vocabulary
5	dino_vlad_viz_layers.py	Visualize the DINO Vocabulary for different ViT layers
6	dino_gp.py	Group pooling over DINO ViT features
7	dino_vlad_contrastive_train.py	DEPRECATED: Contrastive training over Dino VLAD descriptors
8	dino_multilayer_vlad.py	DEPRECATED: VLAD over multiple DINO ViT layer features
9	dino_attnetion.py	DEPRECATED: Visualize Dino attentions

CLIP

S. No.	File	Description
1	clip_top_k_vpr.py	CLIP features for VPR
2	patch_clip.py	CLIP on patches of image -> VLAD

CosPlace

S. No.	File	Description
1	cosplace_vit_vlad.py	Extract features and do VLAD from CosPlace features (ViT backbone)

Dataset Cluster Visualization

For the cluster projections of the datasets

S. No.	File	Description
1	custom_gem_pca_clustering.py	PCA clustering over GeM pooled descriptors of CosPlace, MixVPR, and NetVLAD
2	custom_gem_tsne_clustering.py	t-SNE clustering over GeM pooled descriptors of CosPlace, MixVPR, and NetVLAD
3	dino_v2_datasets_gem_pca_clustering.py	PCA clustering over GeM pooled DINOv2 ViT features
4	dino_v2_datasets_gem_tsne_clustering.py	t-SNE clustering over GeM pooled DINOv2 ViT features
5	dino_v2_datasets_tsne_clustering.py	Dataset visualizations using t-SNE over DINOv2 ViT features (sub-sample all patches across all images)

Bash Scripts

For bulk runs

S. No.	File	Description
1	clip_ablations.sh	Run dataset ablations for CLIP
2	cosplace_vit_ablations.sh	Ablations for CosPlace ViT layer features (VLAD aggregation)
3	dino_gem_ablations.sh	DINO (v1) GeM ablations
4	dino_global_ablations.sh	DINO (v1) global (CLS token) ablations
5	dino_global_vocab_vlad_ablations.sh	DINOv2 VLAD (using global map & domain-specific vocabulary) ablations
6	dino_gp_ablations.sh	DINO (v1) group pooling (max and average) ablations
7	dino_multilayer_vlad_ablations.sh	DINO (v1) multi-layer (ViT feature) VLAD ablations
8	dino_v2_gem_ablations.sh	DINOv2 (ViT features) GeM ablations
9	dino_v2_global_ablations.sh	DINOv2 global feature (CLS token) ablations
10	dino_v2_global_vocab_vlad_ablations.sh	Ablations over global vocabulary (cluster centers) made using DINOv2 VLAD (ViT features)
11	dino_v2_gp_ablations.sh	Ablations for DINOv2 group pooling (max and average) over ViT features
12	dino_v2_vlad_ablations.sh	Ablations for DINOv2 VLAD (ViT features)
13	dino_vlad_ablations.sh	DINO (v1) VLAD (ViT features) ablations