Textbook of AI

Also known as: AUC-ROC, AUROC

AUC stands for Area Under the Curve, and almost always refers to the area under the Receiver Operating Characteristic (ROC) curve, which plots the true-positive rate against the false-positive rate as the classification threshold varies. AUC-ROC ranges from 0.5 (random guessing) to 1.0 (perfect classification); a classifier with AUC 0.8 correctly ranks a random positive example above a random negative example 80% of the time.

AUC has several attractive properties. It is threshold-independent, summarising the classifier's performance across all possible operating points. It is invariant to class proportion within reason, making it useful when the deployment class balance differs from training. It has a clean probabilistic interpretation as the probability of correct pairwise ranking. For these reasons, AUC is one of the most widely reported binary classification metrics.

However, AUC can be misleading under severe class imbalance. When positives are rare, the ROC curve is dominated by the large negative class, and even modest true-positive rates produce high AUC even though most positive predictions are wrong. In such cases, the area under the precision-recall curve (AUC-PR or average precision) is more informative, since it focuses on the positive class. Reporting both metrics, along with precision, recall, and F1 at a chosen threshold, gives the most complete picture of classifier performance.