DiffBIR v2

News About Updated Manuscript 📖

• Rename LAControlNet (a little confusing😆) to IRControlNet.
• We train IRControlNet on our filtered laion2b-en dataset, which contains around 15M high-quality images. The pretrained weight is available now.
• We compare IRControlNet with 6 model variants and find that IRControlNet is good enough as a backbone for generation module.
• We support three BIR tasks: BSR, BFR and BID (Blind Image Denoising), while different tasks share the same IRControlNet. Visual examples can be found here.
• During inference, we directly use off-the-shelf restoration models for degradation removal. More details can be found here.
• We propose region-aware restoration guidance to better achieve a trade-off between quality and fidelity.

News About Code Base 👨‍💻

keep it simple and stupid.

• Free from pytorch lightning and LDM code base. Now the code has been rearranged as simple as possible.
  • Lightning modules have been deleted.
  • Put all model-related code (UNet, VAE, CLIP, etc.) to a single directory.
  • Provide two minimal training scripts for training stage1 and stage2 model, built upon accelerate with the simplest training-loop style.
• Upgrade pytorch to 2.2.2 for 1) built-in sdp attention 2) torch.compile.
• Copy the clip-related code from open-clip. Now chinese users can free from the warning of failed connection with Hugging Face.
• Only save the paramters of IRControlNet, which reduces the size of checkpoint from 9GB to 1GB.