Textbook of AI

Supervised Learning is the machine learning paradigm in which an algorithm learns a function from a training set of input–output pairs ${(x_i, y_i)}_{i=1}^n$. The "supervision" lies in the fact that the correct answer—the label or target—is provided for every training example, allowing the model to measure its errors and adjust accordingly. Once trained, the model can produce predictions for new, unseen inputs.

Supervised learning problems divide into two main categories. In classification, the output is a discrete category: is this email spam or not, which of ten digits does this image show, what disease does this X-ray indicate. In regression, the output is a continuous value: tomorrow's temperature, a house's selling price, a patient's blood-pressure response to a drug. Loss functions differ accordingly: cross-entropy for classification, mean squared error for regression.

Most of the commercially successful AI systems deployed today—spam filters, fraud detection, recommendation systems, medical image analysis, speech recognition—are built on supervised learning. Its great strength is that, given enough high-quality labelled data, even relatively simple algorithms can produce highly accurate predictions. Its great weakness is precisely this dependence on labelled data, which is often expensive, time-consuming, and sometimes impossible to obtain at the scale modern models demand.