While most vectors twist when a matrix is applied, eigenvectors only stretch along their own axis.
From the chapter: Chapter 2: Linear Algebra
Glossary: eigenvector, eigenvalue
Transcript
Apply a matrix to a vector and most of the time it twists. The direction changes.
Watch what happens as we sweep a vector around the circle and apply the same matrix at every angle. Every output points somewhere new.
Except at certain special directions. Here, and again here. The output points the same way as the input, only longer or shorter.
These are the eigenvectors of the matrix. The directions that the matrix only stretches, never rotates.
The amount each eigenvector stretches by is its eigenvalue. A two by two matrix typically has two such directions, and two such stretch factors.
Why do they matter. Because if you write any vector as a sum of eigenvectors, the matrix's action becomes one number times each piece. The whole transformation collapses to scaling.
Powers of the matrix multiply each scale repeatedly. This is why eigenvalues control whether systems blow up, decay, or oscillate.
Principal component analysis, spectral methods, the stability of recurrent networks, the convergence of optimisers. All of them lean on this single picture: the matrix has favourite directions, and on those directions it merely scales.