Looking ahead
The arc of generative models, from Boltzmann machines through GANs to diffusion, is in many respects the arc of modern AI: a few mathematical insights, Markov chain Monte Carlo, the variational lower bound, score matching, the change of variables, repeatedly reformulated and re-engineered until they scaled. The current dominance of diffusion models is unlikely to be permanent. Already in 2024–2025, autoregressive image and video models, transformers operating directly on patch tokens, and unified score-based models are competing for the next decade. What stays constant are the fundamentals derived in this chapter: the chain rule of probability, the ELBO, the change of variables, the score function. Master those, and any new architecture is a small intellectual jump.
The next chapter, Chapter 15, Modern AI Systems, turns to how these generative models are deployed: alignment, retrieval-augmented generation, agentic systems, and the engineering stack that makes a 100-billion-parameter language model usable on a smartphone.