L1 makes a diamond, L2 a circle, L-infinity a square. Each defines distance differently.
From the chapter: Chapter 2: Linear Algebra
Glossary: norm, vector space
Transcript
A norm assigns a length to every vector. The unit ball is the set of vectors with length one.
The L2 norm, Euclidean distance. Square the entries, sum, take the square root. Its unit ball is the familiar circle. Distances are straight-line distances.
The L1 norm. Sum the absolute values. Its unit ball is a diamond, rotated forty-five degrees. Distances follow city blocks.
The L-infinity norm. Take the largest absolute value. Its unit ball is a square. Distances are the worst-case axis distance.
These are not interchangeable. L1 regularisation pushes solutions to the diamond's corners, where most coordinates are zero. That is why Lasso produces sparse models.
L2 regularisation pulls solutions toward the origin without preferring any axis. Ridge regression shrinks all coefficients smoothly.
L-infinity matters in adversarial robustness, where the attacker is bounded by per-pixel perturbation.
Different norms encode different geometric intuitions. The choice changes which vectors count as close, and which solutions an optimiser prefers.