Multivariate Taylor Approximations & Hessian Classification
“Multivariate Taylor series, Hessian matrix H, critical points, eigenvalues, and the discriminant D = f_xx f_yy - (f_xy)^2.”
1.Hessian Curvature & Classification of Multivariable Critical Points
§ 2.1.To optimize multivariable loss functions L(w_1, ..., w_d), we locate critical points where grad(f) = 0.
The Hessian Matrix H:
Captures all second partial derivatives:
H = [ [d^2 f/dx_1^2, d^2 f/dx_1 dx_2], [d^2 f/dx_2 dx_1, d^2 f/dx_2^2] ].
By Clairaut's theorem, d^2 f / dx_i dx_j = d^2 f / dx_j dx_i, so H is always symmetric.
Local Curvature Classification:
Near critical point x, f(x + Delta x) - f(x) ≈ (1/2) Delta x^T H(x) Delta x.
- If H > 0 (Positive Definite), quadratic form is strictly positive in all directions => Local Minimum.
- If H < 0 (Negative Definite), quadratic form is strictly negative => Local Maximum.
- If H has both positive and negative eigenvalues => Saddle Point.
2. Worked Practice Problem: Complete Critical Point Classification
Evaluate both critical points separately; check discriminant D and f_xx at each point.
Problem: Find and classify all critical points of f(x, y) = x^3 + y^3 - 3xy.
Step 1: Compute first partial derivatives and set to zero:
df/dx = 3x^2 - 3y = 0 => y = x^2
df/dy = 3y^2 - 3x = 0 => x = y^2
Step 2: Solve the simultaneous system:
Substitute y = x^2 into x = y^2:
x = (x^2)^2 = x^4 => x^4 - x = 0 => x * (x^3 - 1) = 0.
Real solutions: x = 0 or x = 1.
If x = 0 => y = 0^2 = 0. Point 1: (0, 0).
If x = 1 => y = 1^2 = 1. Point 2: (1, 1).
Step 3: Compute second partial derivatives (Hessian entries):
f_xx = 6*x
f_yy = 6*y
f_xy = -3
Discriminant D(x, y) = f_xx f_yy - (f_xy)^2 = (6x)(6y) - (-3)^2 = 36xy - 9.
Step 4: Classify Point 1 at (0, 0):
D(0, 0) = 36(0)(0) - 9 = -9 < 0.
Since D < 0, Hessian is indefinite.
Conclusion for (0, 0): SADDLE POINT with value f(0, 0) = 0.
Step 5: Classify Point 2 at (1, 1):
D(1, 1) = 36(1)(1) - 9 = 27 > 0.
f_xx(1, 1) = 6(1) = 6 > 0.
Since D > 0 and f_xx > 0, Hessian is Positive Definite.
Conclusion for (1, 1): LOCAL MINIMUM with value f(1, 1) = 1^3 + 1^3 - 3(1)(1) = -1.
Summary: (0, 0) is a saddle point; (1, 1) is a strict local minimum.
Primary Academic References & Exam Sources
Every concept in Notes From a B.Tech Brain is synthesized from authoritative academic texts, seminal peer-reviewed research, verified exam archives, and renowned university lectures.