Textbook of AI

A feedback loop is a circular causal structure in which a portion of a system's output is routed back to influence its own future input. The two qualitatively distinct regimes are negative feedback, which stabilises the system around a set-point, and positive feedback, which amplifies deviations and drives the system towards extremes.

The thermostat is the canonical negative-feedback example: a temperature sensor compares the current room temperature with a target, and the controller increases heat output when the room is colder than the target and reduces it when it is warmer. The deviation drives a corrective response that pulls the deviation back towards zero, producing stability. Cruise control in cars, central heating, blood-glucose homeostasis, and the autopilot of an aircraft are all instances of the same logical structure.

Positive feedback does the opposite: a deviation drives a response that increases the deviation. Microphone howl (sound from a speaker re-enters the microphone, is re-amplified, fed back to the speaker, and so on) is the textbook example. Population growth in the absence of resource limits, the explosive yield of a critical fission reaction, the accumulation of viral copies in an infected cell, and the buying spirals of a financial bubble are all positive-feedback phenomena. Untempered positive feedback is unstable; useful systems mix the two, e.g. action potentials in neurons combine fast positive feedback (sodium channels open and depolarise the membrane) with slower negative feedback (potassium channels open and repolarise) to produce the characteristic spike.

Cybernetics and control theory

Feedback was the central theoretical device of cybernetics, formalised by Norbert Wiener in Cybernetics: Or Control and Communication in the Animal and the Machine (1948), and is the engineering substrate of every automatic control system. PID controllers, combining proportional, integral and derivative terms, are the workhorse of industrial process control:

$$u(t) = K_p e(t) + K_i \int_0^t e(\tau) d\tau + K_d \frac{de(t)}{dt},$$

where $e(t)$ is the error signal. Modern control theory generalises with state-space models, optimal control (LQR, LQG), robust control ($H_\infty$), and model-predictive control, all of which stabilise systems whose dynamics are too uncertain or too complex to control open-loop.

In machine learning

The agent–environment interaction of reinforcement learning is fundamentally a feedback loop: the agent's actions shape the states and rewards it subsequently observes, and learning closes the loop by updating the policy in response to the rewards received. The Bellman equation is the mathematical expression of this circularity: a value depends on the values of successor states.

Modern AI safety is concerned with feedback loops at an entirely different level of abstraction, between deployed models and the data they generate (model-collapse risk as future training corpora contain more and more model-produced text), between recommender systems and the preferences they help shape (a recommender that learns from clicks ends up training users' click behaviour), between AI assistants and the documentation written about them (which is then ingested into the next round of training). These systemic, sociotechnical feedback loops are difficult to characterise mathematically but are increasingly central to the field.