Reproduction issues for pretrained Kinetics models

[LukasHedegaard]

Hi, thanks for sharing the project codebase! 😊

I'm having issues reproducing any sort of results on Kinetics using your pretrained models.

So far, I have:

• Downloaded Kinetics600 using the crawler
• Generated the annotation file using utils/kinetics_json.py
• Run run-kinetics.sh:

python main.py --root_path '' \
 	--video_path ~/datasets/Kinetics \
 	--annotation_path Efficient-3DCNNs/annotation_Kinetics/kinetics.json \
 	--result_path Efficient-3DCNNs/results \
 	--resume_path Efficient-3DCNNs/results/kinetics_mobilenetv2_0.45x_RGB_16_best.pth \
	--dataset kinetics \
 	--sample_size 112 \
 	--n_classes 600 \
 	--model mobilenetv2 \
 	--version 1.1 \
 	--groups 3 \
 	--width_mult 0.45 \
 	--train_crop random \
 	--learning_rate 0.1 \
 	--sample_duration 16 \
 	--batch_size 16 \
 	--n_threads 16 \
 	--checkpoint 1 \
 	--n_val_samples 1 \
	--no_train \
 	--no_val \
 	--test

• Gathered result using python utils/video_accuracy.py
• Repeated the above steps for other models (e.g. kinetics_shufflenet_1.0x_G3_RGB_16_best.pth with --model shufflenet and width_mult 1.0)
• Repeated the above steps using python test_models.py
  • fixing a few bugs in lines
    • 56: correct_k = correct[:k].float().sum().item(),
    • 104: temporal_transform = TemporalCenterCrop(opt.sample_duration, opt.downsample)
    • 121: assert opt.model == checkpoint['arch']
• Added in combinations of --std_norm, --norm_value 255, and --no_mean_norm.

However, all I get are random predictions (~0.001 acc).

Do you have a working example for the pretrained Kinetics models, you could share?

Thanks in advance 😊