Inner Products, Gram-Schmidt & QR Factorization
“Orthogonal bases, Gram-Schmidt process, QR decomposition A = QR, and least-squares projection matrices.”
1.Orthogonal Geometry, Gram-Schmidt & QR Decomposition
§ 4.1.Two vectors are orthogonal if their inner product is zero: u . v = 0. An orthonormal set consists of mutually orthogonal unit vectors.
The Gram-Schmidt Process:
Given a linearly independent basis {v1, v2, ..., vk}, construct an orthonormal basis {e1, e2, ..., ek}:
u1 = v1, e1 = u1 / ||u1||.
u2 = v2 - (v2 . e1) * e1, e2 = u2 / ||u2||.
u_k = v_k - Sum_{j=1}^{k-1} (v_k . e_j) * e_j, e_k = u_k / ||u_k||.
QR Factorization:
A = Q R where Q has orthonormal columns and R is upper triangular with r_{ij} = v_j . e_i.
In least-squares regression, substituting A = Q R into A^T A x_hat = A^T b simplifies the normal equations to R x_hat = Q^T b, which can be solved instantly via back-substitution without matrix inversion.
2. Worked Practice Problem: 3D Gram-Schmidt Orthonormalization
Ensure intermediate vectors u2 and u3 are strictly orthogonal to all previously constructed vectors before normalizing.
Problem: Orthonormalize the basis { v1 = [1, 1, 0]^T, v2 = [1, 0, 1]^T, v3 = [0, 1, 1]^T } in R^3.
Step 1: First orthonormal vector e1:
u1 = v1 = [ 1, 1, 0 ]^T.
||u1|| = sqrt(1^2 + 1^2 + 0^2) = sqrt(2).
e1 = [ 1/sqrt(2), 1/sqrt(2), 0 ]^T.
Step 2: Second orthogonal vector u2 and normalized e2:
u2 = v2 - (v2 . e1) * e1.
v2 . e1 = (1)(1/sqrt(2)) + (0)(1/sqrt(2)) + (1)(0) = 1/sqrt(2).
(v2 . e1) e1 = (1/sqrt(2)) [ 1/sqrt(2), 1/sqrt(2), 0 ]^T = [ 1/2, 1/2, 0 ]^T.
u2 = [ 1, 0, 1 ]^T - [ 1/2, 1/2, 0 ]^T = [ 1/2, -1/2, 1 ]^T.
||u2|| = sqrt( (1/2)^2 + (-1/2)^2 + 1^2 ) = sqrt( 1/4 + 1/4 + 1 ) = sqrt(6/4) = sqrt(6)/2.
e2 = u2 / ||u2|| = (2/sqrt(6)) * [ 1/2, -1/2, 1 ]^T = [ 1/sqrt(6), -1/sqrt(6), 2/sqrt(6) ]^T.
Step 3: Third orthogonal vector u3 and normalized e3:
u3 = v3 - (v3 . e1)e1 - (v3 . e2)e2.
v3 . e1 = 0 + 1/sqrt(2) + 0 = 1/sqrt(2) => (v3 . e1)*e1 = [ 1/2, 1/2, 0 ]^T.
v3 . e2 = 0 - 1/sqrt(6) + 2/sqrt(6) = 1/sqrt(6) => (v3 . e2)e2 = (1/sqrt(6)) [ 1/sqrt(6), -1/sqrt(6), 2/sqrt(6) ]^T = [ 1/6, -1/6, 2/6 ]^T.
u3 = [ 0, 1, 1 ]^T - [ 1/2, 1/2, 0 ]^T - [ 1/6, -1/6, 1/3 ]^T
u3 = [ -2/3, 2/3, 2/3 ]^T.
Verify orthogonality: u3 . u1 = -2/3 + 2/3 + 0 = 0; u3 . u2 = -1/3 - 1/3 + 2/3 = 0. (Strictly orthogonal!).
||u3|| = sqrt( 4/9 + 4/9 + 4/9 ) = sqrt(12)/3 = 2/sqrt(3).
e3 = u3 / ||u3|| = [ -1/sqrt(3), 1/sqrt(3), 1/sqrt(3) ]^T.
Conclusion: Orthonormal basis is { [1/sqrt(2), 1/sqrt(2), 0]^T, [1/sqrt(6), -1/sqrt(6), 2/sqrt(6)]^T, [-1/sqrt(3), 1/sqrt(3), 1/sqrt(3)]^T }.
Primary Academic References & Exam Sources
Every concept in Notes From a B.Tech Brain is synthesized from authoritative academic texts, seminal peer-reviewed research, verified exam archives, and renowned university lectures.