Textbook of AI

Also known as: belief network, causal network

A Bayesian network, introduced by Judea Pearl in Probabilistic Reasoning in Intelligent Systems (1988), is a directed acyclic graph (DAG) whose nodes are random variables and whose edges represent direct conditional dependencies. Each node has a conditional probability table (or, for continuous variables, a conditional density) specifying its distribution given its parents. The joint distribution factorises as the product of these conditionals.

Bayesian networks compactly encode the conditional independencies of a joint distribution: a variable is conditionally independent of its non-descendants given its parents. Inference, computing posterior distributions of unobserved variables given evidence, can in some structures be performed efficiently by Pearl's belief propagation algorithm; for general structures it is NP-hard, motivating approximate methods including loopy belief propagation, variational inference, and Markov chain Monte Carlo.

The framework reshaped both AI and statistics. Hidden Markov models are Bayesian networks with a particular sequential structure; Kalman filters are continuous Bayesian networks for linear-Gaussian dynamics; probabilistic relational models, dynamic Bayesian networks and latent Dirichlet allocation are all instances of the framework. In the 2000s, Bayesian networks dominated AI's treatment of uncertainty before being partly displaced by deep learning's implicit, learned representations of probability.

Pearl's later work extended the framework to causal inference: with the addition of an intervention operator (the do-operator), Bayesian networks become structural causal models capable of representing causation rather than mere correlation.