Orthogonal Projection

Precise definition

For nonzero $u$, the orthogonal projection of $v$ onto $\operatorname{span}\{u\}$ is $\operatorname{proj}_u v=(v\cdot u)/(u\cdot u)\,u$. If columns of $Q$ are orthonormal, projection onto their span is $QQ^Tv$. For a full-column-rank $A$, it is $A(A^TA)^{-1}A^Tv$.

Notation and mathematical language

The decomposition $v=p+r$ has $p$ in the subspace and residual $r$ perpendicular to it. This $p$ uniquely minimizes distance $\|v-w\|$ over all subspace vectors $w$.

Conceptual picture

Projection keeps the component aligned with the target subspace and discards the orthogonal component. The projection matrix $P$ satisfies $P^2=P$ and, for orthogonal projection, $P^T=P$.

Conditions and key results

The target must be a subspace for a linear projection through the origin. Projection onto an affine set requires a translation. The simple sum $\sum(v\cdot u_i)u_i$ assumes the $u_i$ are orthonormal; otherwise use denominators or solve a Gram system.

A reliable strategy

1. Specify the inner product and target subspace; check whether a supplied basis is orthogonal or orthonormal.
2. Compute projection coefficients with the correct norm denominators or use $QQ^T$.
3. Form $p$ and residual $r=v-p$.
4. Verify membership, orthogonality, and the idempotence or closest-point property.

Fully worked example

Interpretation and application

Projection powers least squares, signal decomposition, graphics, and approximation. The notion of 'closest' depends on the selected inner product and scaling, so weighted applications may require a weighted projection.

Common mistakes

Verification and reasonableness

• Confirm $p$ is a combination of target basis vectors.
• Compute inner products of $r$ with every basis vector.
• Apply the projection twice and verify the result is unchanged.

Practice

1. Project $(4,3)$ onto the $x$-axis.
2. What characterizes the residual?
3. What identities hold for an orthogonal projection matrix?

Further deduction

For complementary orthogonal subspaces $W$ and $W^\perp$, every vector has a unique decomposition $v=\operatorname{proj}_Wv+\operatorname{proj}_{W^\perp}v$. Pythagoras gives $\|v\|^2=\|\operatorname{proj}_Wv\|^2+\|v-\operatorname{proj}_Wv\|^2$. This identity proves the closest-point property without calculus.

Projection onto the solution set of linear constraints can use Lagrange multipliers. To project $v$ onto $\{x:Cx=0\}$ when rows of $C$ are independent, subtract the row-space component: $p=v-C^T(CC^T)^{-1}Cv$. The residual lies in $\operatorname{Row}C$, while $Cp=0$ verifies feasibility.

Projection onto an affine set $a+U$ is obtained by translating, projecting, and translating back: $\operatorname{proj}_{a+U}(v)=a+\operatorname{proj}_U(v-a)$. Unlike a subspace projection, this map need not send zero to zero and is generally affine rather than linear. The residual is orthogonal to the direction subspace $U$, which yields the closest-point proof. This formulation covers fitting a point to a line or plane that does not pass through the origin.

Related topics

• Orthogonality
• Least Squares
• Gram-Schmidt Process