Different fairness criteria pull a classifier in different directions, and they cannot all hold at once.
From the chapter: Chapter 16: Ethics & Safety
Glossary: demographic parity, fairness
Transcript
A classifier predicts who gets a loan. Two protected groups, A and B, with different base rates of default.
Demographic parity says: the approval rate must be equal across the groups. The same fraction of A applicants and B applicants get loans.
But the groups have different default rates. To match the approval rates, the bank approves some risky A applicants it would have declined if they were B, and vice versa. Calibration breaks.
Equalised odds says: among applicants who would default, the false-positive rate must match across groups. Among applicants who would repay, the true-positive rate must match. The classifier's errors are distributed equally.
Calibration says: among applicants given a 70 percent approval score, the actual repayment rate must be 70 percent in both groups.
Watch the rates as we slide the decision threshold. We can hit demographic parity, or equalised odds, or calibration, but not all three at once. A formal impossibility theorem says: if base rates differ between groups, you must give one of them up.
Fairness, then, is not one criterion but a family of criteria, each capturing a different intuition about what counts as fair. Choosing among them is a value judgement, not a technical problem.