Textbook of AI

Control theory is the engineering and mathematical study of dynamical systems with inputs (controls) and outputs, designed to maintain desired behaviour despite disturbances and uncertainty. It is closely related to cybernetics (Wiener, 1948), from which much modern control theory descends, and is the engineering counterpart of reinforcement learning in artificial intelligence.

The classical setting models a plant by a state-space system

$$\dot{\mathbf{x}}(t) = f(\mathbf{x}(t), \mathbf{u}(t)) + \mathbf{w}(t), \qquad \mathbf{y}(t) = h(\mathbf{x}(t)) + \mathbf{v}(t),$$

where $\mathbf{x}$ is the state, $\mathbf{u}$ the control input, $\mathbf{y}$ the measured output, and $\mathbf{w}, \mathbf{v}$ are process and measurement noise. For linear systems, $f(\mathbf{x}, \mathbf{u}) = A\mathbf{x} + B\mathbf{u}$ and frequency-domain analysis via the Laplace transform gives the transfer function and tools like Bode, Nyquist and root-locus plots.

Key concepts:

• Open-loop control: the input depends only on the reference, not on the output. Brittle to disturbances and modelling error.
• Closed-loop / feedback control: the input depends on the error between desired and actual output. Robust to disturbances; the foundation of every modern control system.
• PID controller: $u(t) = K_p\, e(t) + K_i \int_0^t e(\tau)\, d\tau + K_d\, \dot{e}(t)$. The proportional--integral--derivative controller is the workhorse of industrial control, found in everything from cruise control to rocket gimbals.
• Stability: a system's tendency to return to equilibrium after disturbance. Analysed via Lyapunov functions, eigenvalues of $A$, or Nyquist criteria.
• Controllability and observability: structural conditions (Kalman) determining whether the state can be steered from any point to any other (controllability) and inferred from outputs (observability).
• Linear-quadratic regulator (LQR): optimal control for linear dynamics with quadratic cost $\int (\mathbf{x}^\top Q \mathbf{x} + \mathbf{u}^\top R \mathbf{u})\ , dt$. Solved in closed form by the algebraic Riccati equation.
• Kalman filter: optimal recursive state estimator for linear Gaussian systems; combined with LQR yields the LQG controller.
• Model predictive control (MPC): solve a finite-horizon optimisation each timestep, apply the first action, re-plan. Handles constraints naturally; ubiquitous in chemical processes, autonomous vehicles, drone flight.
• Robust control ($H_\infty$, $\mu$-synthesis): design controllers that work despite bounded uncertainty in the plant.
• Adaptive control (MRAC, self-tuning regulators): controllers that learn the plant dynamics online while controlling.
• Nonlinear control: feedback linearisation, sliding-mode control, backstepping, contraction analysis.

Connection to AI and machine learning: reinforcement learning is essentially adaptive optimal control where the dynamics and cost function are unknown and must be learned from interaction. Modern deep RL has deep roots in control theory: temporal-difference learning descends from Werbos's adaptive critic; value functions and dynamic programming are Bellman's; policy gradients generalise the Pontryagin maximum principle. Differentiable control (e.g. DDPG, MPPI, SAC) and differentiable simulators blend classical control with deep learning. Conversely, control-theoretic tools -- Lyapunov stability, contraction analysis, $H_\infty$ robustness -- increasingly appear in analyses of safe RL and certified neural control. Modern robotics, autonomous driving, drone autopilots and humanoid bipedal walking sit squarely at the intersection.