The generator tries to fool the discriminator. The discriminator tries not to be fooled. Equilibrium is realistic samples.
From the chapter: Chapter 14: Generative Models
Glossary: generative adversarial network, gan
Transcript
A generative adversarial network is a two-player game between two neural networks.
The generator takes random noise and outputs an image. Initially garbage.
The discriminator looks at an image and decides: real, drawn from the training set, or fake, drawn from the generator. Initially also clueless.
Training alternates. Show the discriminator a real image and a fake image. It learns to distinguish them. Its gradient pushes its weights toward better discrimination.
Then show the generator a noise sample. It produces a fake. Run the fake through the discriminator. The generator's gradient is the opposite of the discriminator's: it adjusts to fool the discriminator.
Watch the loss curves. They oscillate. As the generator improves, the discriminator faces a harder task. As the discriminator improves, the generator faces a harder task.
In the ideal equilibrium, the generator's distribution matches the data distribution exactly. The discriminator is at fifty percent everywhere.
In practice, training is famously unstable. Mode collapse: the generator finds one trick the discriminator cannot see and produces the same output for every noise input. Vanishing gradients: the discriminator wins too quickly and gives the generator no signal.
Tricks like Wasserstein loss, gradient penalty, spectral normalisation, and progressive growing made GANs train reliably and produced photorealistic faces by the late 2010s.