What does compute_elbo_with_multiple_samples compute?

[christophfeinauer]

Hi, I was looking at the function compute_elbo_with_multiple_samples in pfam_msa/script/VAE_model.py.

I am a bit confused about what it is calculating: Looking at the line log_weight = (log_Pxz - log_QzGx).detach().data
it seems to me that it is calculating an approximation to

log E_q(z|x) [ p(x,z) / q(z|x) ] = log E_p(z) [ p(x|z) ] = log p(x),

which is the log probability of x and not the ELBO.

Is that correct or is there something I am not understanding?

[xqding]

It can be understood in both ways: 1. approximation of log p(x) as you have shown. 2. an ELBO based on multiple samples (regular ELBO in vanilla VAE is based on one sample). You can see this in Equation 9 of the Importances Weighted Auto-Encoder paper.