Orthogonal Projection

A precise treatment of projections onto lines and subspaces, residual orthogonality, and projection matrices.

Orthogonal projection separates a vector into fitted and perpendicular components. A vector is decomposed into a component along a subspace and a right-angled residual. Basis arrows show how coordinates reconstruct the projected result.

Precise definition

For nonzero $u$, the orthogonal projection of $v$ onto $\operatorname{span}\{u\}$ is $\operatorname{proj}_u v=(v\cdot u)/(u\cdot u)\,u$. If columns of $Q$ are orthonormal, projection onto their span is $QQ^Tv$. For a full-column-rank $A$, it is $A(A^TA)^{-1}A^Tv$.

Notation and mathematical language

The decomposition $v=p+r$ has $p$ in the subspace and residual $r$ perpendicular to it. This $p$ uniquely minimizes distance $\|v-w\|$ over all subspace vectors $w$.

Conceptual picture

Projection keeps the component aligned with the target subspace and discards the orthogonal component. The projection matrix $P$ satisfies $P^2=P$ and, for orthogonal projection, $P^T=P$.

Fully worked example

Interpretation and application

Projection powers least squares, signal decomposition, graphics, and approximation. The notion of 'closest' depends on the selected inner product and scaling, so weighted applications may require a weighted projection.