Evaluation results

[Sharpiless]

python test_scripts/BI/REDS/test_IART_REDS4_N16.py

2024-06-27 16:32:08,597 INFO: Average PSNR: 32.303768 dB for 30 clips. 
2024-06-27 16:32:08,597 INFO: Average SSIM: 0.899245  for 30 clips. 

Awesome work! But i get 32.30 on REDS, which mismatches the results in the paper. Any suggestions?

[kai422]

Hi,

Can you check if your cuda version is 11? I found this affected the testing results in my side.

Regards,
Kai

[Sharpiless]

Hi, thanks for reply! I'm using python3.8 pytorch 1.13.1+cu117 (system cuda 11.3) and 4090 gpus. I'll check it on another machine,

[xinyuliu-jeffrey]

On python 3.9 + cuda 12.2, the results match the results for the "python test_IART_REDS4_N16.py"

2024-07-17 10:36:10,772 INFO: Average PSNR: 32.899396 dB for 4 clips.
2024-07-17 10:36:10,772 INFO: Average SSIM: 0.913785 for 4 clips.

However I wonder if the table means two different models are used, with the REDS_N16 model evaluated on REDS4, and the finetuned Vimeo model evaluated on Vimeo-90K-T and Vid4. That is, the IA-CNN for Vimeo and Vid4 evaluation is finetuned from the N15 model trained on REDS?