The Wiki
Advanced7 min read

Adversarial duel

Two networks fight: a forger against a detector. The generator learns to produce real-looking data because losing to a critic is still training signal.

The VAE's blur is a direct cost of defining what "real-looking" means only through pixel reconstruction. The GAN fix (Goodfellow et al., 2014): don't define it — let a second network learn it. Two players, one minimax game:

min⁡Gmax⁡D  [Ereallog⁡D(x)+Ezlog⁡(1−D(G(z)))] \min_G \max_D \; \Big[ \mathbb{E}_{\text{real}} \log D(x) + \mathbb{E}_{z}\log (1 - D(G(z))) \Big]
  • Discriminator DD: real or fake? Trained to detect.
  • Generator GG: maps noise to samples; trained to fool — and DD's gradient w.r.t. the sample tells GG what would make it look real.

A critic you never had to write

The VAE's loss (pixel MSE) penalizes wrong pixels — that's why frames blur. The GAN's discriminator penalizes wrongness of structure, whatever features a trained network notices as fake. Every time the critic improves its eye, the forger is graded against a sharper version of "what looks real". Generated images become crisp because crispness is what detects.

Grey: real data ring. Purple: generator’s fakes, learning to imitate under the critic’s eye.

fooling rate: 0%

Press run: the critic (ring distance) instantly punishes off-ring points, and the generator reshapes itself against that pressure.

Honest two-player dynamics at toy scale: the critic’s notice is the ring-fit metric, the generator integrates its pull. Real GANs use learned critics and convolutional stacks; the balance/collapse/oscillation patterns shown are the real dynamics — no supervised loss anywhere here.

Why the game is hard to run

Minimax two-player training is unstable in ways one-loss training never is:

  1. Balance. If DD gets too good, log⁡(1−D(G(z)))\log(1 - D(G(z))) saturates — the forger's gradient vanishes and learning stalls. If DD is too weak, GG's updates chase a useless target. Most training failures are this balance; their signature is sudden dives and spikes in the loss curves.
  2. Mode collapse. A lazy GG discovers one image that always fools this DD and outputs it (or a few of them) every time — diversity destroyed. This is the GAN failure you will read about; the shown artifact's dials let you feel it.
  3. The arms race never ends. Both models keep changing the definition of "real" under each other — stable equilibrium is not guaranteed for any given configuration, hence the endless practical folklore (spectral norm, feature matching, TTUR).

Sober status: a past champion, not the current

GANs dominated image generation 2014-2020 (StyleGAN's faces were the cultural milestone) and still fit tasks with narrow, dense outputs. For broad image generation they lost — to diffusion — almost entirely, because diffusion supervised every denoising step against the data distribution individually, while the duel only penalized fakes the critic had noticed. But minimax training is a foundational mindset — RLHF is adversarial too: the "critic you never had to write" is the reward model or human vote.

Illustrative vs real

Two small MLPs, real training loop in your browser over 2-D data: the tug-of-war dynamics, mode collapse, and balance-of-fooling are all live. Real GANs run convolutional stacks across image grids with stabilizers — the dynamics are faithfully represented; the image statistics aren't shown.

Where next: diffusion — how today's image generators replaced the duel by supervising every step of a noise-out walk.

This lesson has exercises attached — diagnosing GAN training failures from loss curves — launching once the exercises layer ships.