Textbook of AI

Also known as: inner product, scalar product

The Dot Product (also called inner product or scalar product) of two vectors $\mathbf{u}$ and $\mathbf{v}$ is defined as $\mathbf{u} \cdot \mathbf{v} = \sum_i u_i v_i$. The result is a single number—a scalar, not a vector. The operation is computationally cheap and geometrically rich, making it the workhorse of numerical linear algebra and modern AI.

The geometric interpretation is fundamental: $\mathbf{u} \cdot \mathbf{v} = |\mathbf{u}||\mathbf{v}| \cos\theta$, where $\theta$ is the angle between the vectors. If the dot product is zero, the vectors are orthogonal; if positive, they point in similar directions; if negative, opposite directions. Dividing by the norms yields the cosine similarity, which ranges from $-1$ to $+1$ and is the standard similarity metric in information retrieval, recommendation systems, and embedding-based search.

In machine learning, the dot product is the basic mechanism of similarity and prediction. A linear classifier scores an input $\mathbf{x}$ as $\mathbf{w} \cdot \mathbf{x} + b$. A single neuron in a neural network computes a dot product of its weights with its inputs. Attention in transformers computes dot products between query and key vectors to determine how much one token should attend to another. Matrix multiplication is nothing more than a batched collection of dot products. Understanding the dot product is the single most important step toward fluency in AI mathematics.