Textbook of AI

The Jacobian Matrix is the generalisation of the gradient to vector-valued functions. If $\mathbf{f}: \mathbb{R}^n \to \mathbb{R}^m$, the Jacobian $J$ is the $m \times n$ matrix whose $(i, j)$ entry is $\partial f_i / \partial x_j$. Each row is the gradient of one output component with respect to all inputs. The Jacobian encodes the best linear approximation to $\mathbf{f}$ near a given point—the matrix by which small changes in the input are multiplied to produce (approximate) small changes in the output.

The Jacobian plays a starring role in the multivariate chain rule: the Jacobian of a composition is the product of the Jacobians of the components. In neural network backpropagation, each layer contributes a Jacobian, and the overall gradient is computed by multiplying these together (in practice, by backward-propagating gradient vectors rather than forming full Jacobian matrices).

Jacobians also appear in normalising flows, where the change-of-variables formula $p_x(x) = p_z(f^{-1}(x)) |\det \partial f^{-1}/\partial x|$ requires computing the determinant of a Jacobian. This motivates the design of transformations with triangular or otherwise tractable Jacobians. In robotics and control, the Jacobian relates joint velocities to end-effector velocities, and in sensitivity analysis it quantifies how outputs depend on inputs—a fundamental quantity for uncertainty propagation and adversarial robustness.