Training loss keeps falling. Validation loss bottoms out, then rises. The gap is overfitting.
From the chapter: Chapter 6: ML Fundamentals
Glossary: overfitting, early stopping, validation set, regularisation
Transcript
Every model is fit on a training set, but its real job is to perform well on data it has never seen. The validation set is the proxy for that future data.
Watch the loss curves as a model trains. The training loss, in blue, keeps falling. The model is memorising the training examples.
The validation loss, in red, falls at first because the model is learning patterns that generalise. But after a while it bottoms out and starts to climb. The model has begun to memorise noise that does not exist in the validation set.
The gap between the two curves is the generalisation gap. A small gap means the model is genuinely learning. A large and growing gap means it is overfitting.
Early stopping is the simplest defence. Pick the epoch where validation loss is lowest, and use the model from that point. Everything after is making things worse.
A separate test set, untouched during training and tuning, gives an honest final estimate of how the chosen model will perform in the wild.