Textbook of AI

Connectionism is the school of cognitive science and AI that models cognition as the emergent behaviour of networks of many simple interconnected units, each performing a small local computation. Cognition is understood as the parallel, distributed propagation of activation through such networks; learning is the modification of connection weights.

The position is in tension with the symbolic-AI tradition's view that cognition consists of explicit symbol manipulation according to formal rules. Where symbolic AI sees thinking as proof in a logical calculus, connectionism sees it as relaxation in a high-dimensional continuous space. The two views are not strictly incompatible, symbol manipulation can be implemented in connectionist substrates, and connectionist computation can be analysed in symbolic terms, but historically they have produced different research programmes and different intuitions about what intelligence "really is".

Connectionism's modern incarnation began with the PDP volumes (Rumelhart, McClelland and the PDP Research Group, 1986) and the backpropagation paper of the same year. The 1990s saw a partial retrenchment as symbolic methods (Bayesian networks, statistical NLP) reasserted themselves; the 2010s deep-learning revolution decisively vindicated the connectionist research programme. Modern large language models are connectionist systems of an extreme scale.

Variants and refinements include classical connectionism (the PDP-era position), eliminative connectionism (which holds that symbols are not implemented but eliminated), and hybrid neuro-symbolic approaches (the modern revival of attempts to combine the two paradigms).