Explain VAEs, the ELBO, the KL Term, and Reparameterization

Describe a variational autoencoder, derive the evidence lower bound, and explain why the objective contains a reconstruction term and a KL term. Explain the reparameterization trick and why it enables gradient-based training.

Constraints & Assumptions

• Use a latent-variable model $p(x, z)$, an encoder $q(z \mid x)$, and a decoder $p(x \mid z)$.
• State the prior over $z$ explicitly rather than leaving it implicit.
• Distinguish maximizing the ELBO from exactly maximizing the log evidence.

Clarifying Questions to Ask

• Which likelihood model defines reconstruction quality?
• What posterior family does the encoder produce?
• How is the KL term weighted in the stated variant?

Hint: Add and subtract the posterior. Write $\log p(x)$ as the ELBO plus a nonnegative divergence from the approximate posterior to the true posterior.

What a Strong Answer Covers

• Encoder, decoder, prior, and generative story.
• ELBO derivation and the roles of reconstruction and regularization.
• Closed-form or estimated KL behavior and posterior collapse risk.
• Reparameterization as a deterministic noise transformation and gradient path.

Follow-up Questions

1. What changes in a beta-VAE?
2. Why can a powerful decoder ignore the latent variable?

Overview: Connect VAE components to the ELBO, reconstruction and KL terms, posterior approximation, reparameterized gradients, and collapse risk.