Training code for VAE

[kl2004]

Thanks for the great work and the released code base! 💯

After checking current training code, I notice that the VAE is loading from the pretrained checkpoint vae_ch160v4096z32.pth.

As mentioned #5, the training code for VAE will be released at https://github.com/FoundationVision/vae-pro. Is there a target date for it? I'm very interested in training a VAE model on my custom dataset.

[luohao123]

Same request here, @FoundationVision any plan to release the VAE training code?

[keyu-tian]

@kl2004 @kl2004 we're actively cleaning the codes now. For a temporary reference you can refer to the VQVAE forward at https://github.com/FoundationVision/VAR/blob/main/models/vqvae.py#L56-L59.

[luohao123]

thanks,

def forward(self, inp, ret_usages=False):   # -> rec_B3HW, idx_N, loss
        VectorQuantizer2.forward
        h_BChw, usages, vq_loss, mean_entropy_loss = self.quantize(self.quant_conv(self.encoder(inp)), ret_usages=ret_usages)
        return self.decoder(self.post_quant_conv(h_BChw)), usages, vq_loss, mean_entropy_loss

does this runable code?

[RobertLuo1]

I would like to if the evaluation (e.g. FID LPIPS PSNR, etc.) code of VAE can be released?

[eanson023]

Still looking forward to it...

[ArmeriaWang]

Thanks for the great work @FoundationVision , still looking forward to the training code for multi-res VQVAE!

[FanqingM]

Still lokking forward

[StarCycle]

Still looking forward!

[Junda24]

Still looking forward!

[HalvesChen]

Still looking forward!

[SunzeY]

Still looking forward!

[z379035389]

每天都来看一眼开源没👁️ 👁️

[StarCycle]

泪目啊，啥时候能开源

[keyu-tian]

Dear all @kl2004 @luohao123 @RobertLuo1 @eanson023 @ArmeriaWang @FanqingM @StarCycle @Junda24 @HalvesChen @SunzeY @z379035389

Many thanks for your patience! Our VAE training and inference codebase and model weights would be released next week. Please stay tuned❤️!

I'll post the link here and in the README.

[HalvesChen]

Still looking forward!

[walking-shadow]

Dear all @kl2004 @luohao123 @RobertLuo1 @eanson023 @ArmeriaWang @FanqingM @StarCycle @Junda24 @HalvesChen @SunzeY @z379035389

Many thanks for your patience! Our VAE training and inference codebase and model weights would be released next week. Please stay tuned❤️!

I'll post the link here and in the README.

Still looking forward!

[Junda24]

Still didn't opensouce?

Try ByteDance's work! They have a better work than this

What is the project link？Thanks！

[kl2004]

Hi @keyu-tian, is the VAE training code being released at https://github.com/FoundationVision/vaex?

[mounchiliu]

Still looking forward!

[tomekster]

Hi @keyu-tian, are you still planning to release the training code? Looking forward to it!

[luohao123]

No, not planing.

[lxa9867]

Hi Guys,

I am trying to train the VQVAE (with multi-scale VQ) using the LlamaGen VAE training code. However, I found the codebook utilization rate remains low (about 58%). Does anyone have a good idea about this? Thanks!