Could not reproduce the kitti odometry results

[SenZHANG-GitHub]

I have run CUDA_VISIBLE_DEVICES=3 bash scripts/train_resnet50_pose_256.sh which contains:

TRAIN_SET=data/kitti/kitti_vo_256/
python train.py $TRAIN_SET \
--resnet-layers 50 \
--num-scales 1 \
-b4 -s0.1 -c0.5 --epoch-size 1000 --sequence-length 3 \ 
--with-ssim 1 \ 
--with-mask 1 \
--with-auto-mask 1 \
--with-pretrain 1 \
--log-output \ 
--name resnet50_pose_256

The training process seems good: Eventually we have Total loss: 1.130, photo loss: 1.130, smooth loss: 0.029, consistesh scriptsncy loss: 0.092. Then I following your intruction to run sh scripts/test_kitti_vo.sh where I modify the variables to fit my training path

DATASET_DIR=data/kitti/kitti_odom_test/sequences/
OUTPUT_DIR=vo_results_tmp/
POSE_NET=checkpoints/resnet50_pose_256/07-20-18:24/exp_pose_model_best.pth.tar

However, the evaluation results are quite far away from the reported ones:

Sequence: 9
Tranlsational error (%): 32.49381948479178
Rotational error (deg/100m): 8.876555343671876
ATE (m): 168.36036295068504
RPE (m):  0.2413781835893739                                                                   
RPE (deg):  0.13960476261442967                                                                  
Sequence: 10                                                         
Translational error (%):  12.888807673314357                                                     
Rotational error (deg/100m):  5.588445130500064                          
ATE (m):  31.397818060533346                                                                
RPE (m):  0.0859026892970541                                                                
RPE (deg):  0.11769662553343758  

while the reported numbers of t_err(%)/r_err(degree/100m) for seq 9 and 10 are 7.13/3.05 and 7.79/4.90 respectively.

May I ask whether you have any suggestions on correcting the results? Many thanks!

[JiawangBian]

Odometry results in long sequences are very sensitive, but should not be so bad. Could you check the depth visualization in tensorboard to make sure that the training is correct?