Generative Adversarial Networks

Setup: Assume we have data $x_i$ drawn from distribution $p_{\text{data}}(x)$. Want to sample from $p_{\text{data}}$
Idea: Introduce a latent variable z with simple prior p(z). Sample z ~ p(z) and pass to a Generator Network x = G(z).
Then x is a sample from the Generator distribution $p_G$. Want $p_G=p_{\text{data}}$

Generator Network: Try to fool the discriminator by generating real-looking images
Discriminator Network: Try to distinguish between real and fake images

Jointly train Generator network and Discriminator Network
Train jointly in minimax game

• Discriminator wants to $D(x)=1$ for real data and $D(x)=0$ for fake data
• Generator wants $D(x)=1$ for fake data
  

Train G and D using alternating gradient updates

At start of training, generator is very bad and discriminator can easily tell apart real from fake

• so D(G(z)) close to 0

Architecture: DC-GAN

Generator is an upsampling convolutional network
Discriminator is a convolutional network

Vector Math