Stanford CS236: Deep Generative Models I 2023 I Lecture 9 - Normalizing Flows

Stanford CS236: Deep Generative Models I 2023 I Lecture 9 - Normalizing Flows

Generative Adversarial Networks (GANs)

  • GANs use an adversarial process to train a generator and a discriminator network.
  • The generator network aims to produce realistic samples that can fool the discriminator, while the discriminator network tries to distinguish between real data and samples generated by the generator.
  • GANs offer more flexibility in defining the generative model family since the training objective is not based solely on maximum likelihood.
  • GANs have the advantage of not requiring likelihood evaluations, making them flexible in choosing the generator architecture.
  • Training GANs is challenging due to the Mini-Max optimization problem and convergence issues.

Two-Sample Test

  • Two-sample tests are used to determine whether two groups of samples come from the same probability distribution.
  • A test statistic is used to compare the two groups of samples, and a threshold is set to determine whether to reject the null hypothesis.
  • Choosing a good test statistic is important to minimize type one and type two errors.
  • Instead of handcrafting a test statistic to compare two probability distributions, we can train a classifier (discriminator) to distinguish between samples from the data distribution and samples from the model distribution.
  • The loss of the classifier can be used as a test statistic, where a high loss indicates that the two distributions are different and a low loss indicates that they are similar.

GANs Training

  • The generator is trained to fool the discriminator by minimizing the objective function V.
  • The optimal discriminator is the conditional probability of a point x belonging to the positive class (real samples).
  • The K Divergence is minimized when P(Theta) = P, but in practice, we cannot reach the global optimum.
  • Among suboptimal models, it may be preferable to have one that cannot fool a discriminator over one with high compression.
  • A likelihood-based model can be used as a discriminator, but it defeats the purpose of not needing access to the likelihood.

Overwhelmed by Endless Content?