Skip to content

The goal of this repo is to provide a common evaluation script for image inpainting tasks. It contains some commonly used image quality metrics for inpainting (e.g., L1, L2, SSIM, PSNR and LPIPS).

Notifications You must be signed in to change notification settings

SayedNadim/Inpainting-Evaluation-Metrics

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

22 Commits
 
 
 
 
 
 
 
 

Repository files navigation

I am not maintaining this repo anymore. Please refer to this repo for updated version. Thank you.

Inpainting Evaluation Metrics (On-going)

The goal of this repo is to provide a common evaluation script for image inpainting tasks. It contains some commonly used image quality metrics for inpainting (e.g., L1, L2, SSIM, PSNR and LPIPS).

Pull requests and corrections/suggestions will be cordially appreciated.

Please Note

  • Images are scaled to [0,1]. If you need to change the data range, please make sure to change the data range in SSIM and PSNR.
  • Number of generated images and ground truth images have to be exactly same.
  • I have resized the images to be (256,256). You can change the resolution based on your needs.
  • Please make sure that all the images (generated and gt images) are in the corresponding folders. Currently,it can not calculate metrics if there are sub-folders. I will update the code to calculate for sub-folders as well.
  • LPIPS is a bit slow. So, if you have lots of images, it might take a lot of time. (For ~1000 images, it took around ~15-20 minutes on my personal setup (1 TitanXP). Other metrics are fast and took around ~20 seconds to compute.)

Requirements

Usage

  • Usable Arguments

    • --input_path - path to your generated images (required).
    • --gt_path - path to your ground truth images (required).
    • --batch_size - batch size you want to use (Default to 4).
    • --image_width - width of the image (both generated image and ground truth images will be resized to this width. Default to 256).
    • --image_height - width of the image (both generated image and ground truth images will be resized to this width. Default to 256).
    • --threads - threads to be used for multi-processing (Default to 4).
  • Please provide paths of the folders (i.e., folder of generated images and folder of ground truth images).

    python main.py --input_path path/to/generated/images --gt_path path/to/ground/truth/images

  • If you need to save it in a .txt file, then simply run

    python main.py --input_path path/to/generated/images --gt_path path/to/ground/truth/images >> results.txt

To-do

  • L1
  • L2
  • SSIM
  • PSNR
  • LPIPS
  • FID
  • IS

Acknowledgement

Thanks to PhotoSynthesis Team for the wonderful implementation of the metrics. Please cite accordingly if you use PIQ for the evaluation.

Cheers!!

About

The goal of this repo is to provide a common evaluation script for image inpainting tasks. It contains some commonly used image quality metrics for inpainting (e.g., L1, L2, SSIM, PSNR and LPIPS).

Topics

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages