Drop a perpendicular from a onto b: the dot product is the signed length of the shadow.
From the chapter: Chapter 2: Linear Algebra
Glossary: dot product, vector, projection
Transcript
The dot product of two vectors is one of the most important operations in linear algebra. It has a clear geometric meaning.
Here are two vectors, a and b, drawn from a common origin.
Drop a perpendicular from the tip of a onto the line through b. The shadow that a casts on b is its projection.
The length of this shadow, multiplied by the length of b, is the dot product.
When the angle between a and b is small, the shadow is long and the dot product is large.
When a and b are perpendicular, the shadow has zero length. The dot product is zero.
When a points away from b, the shadow falls on the opposite side. The dot product becomes negative.
In machine learning, the dot product measures alignment between vectors: between an input and a learned weight, between a query and a key, between a sample and a prototype. It is the workhorse of every linear layer in every neural network.