SVM Maximum Margin Hyperplane & Dual Optimization
“Geometric margin 2/||w||, Lagrangian multipliers, KKT complementary slackness, and slack variables.”
1.The Geometry of the Maximum Margin Separator
§ 4.1.Given linearly separable training data with labels y_i in {-1, +1}, infinite hyperplanes w^T x + b = 0 can separate the classes.
The Optimal Margin Criterion:
The distance from any point x to the hyperplane is |w^T x + b| / ||w||. For canonical support vectors on the margin boundary |w^T x + b| = 1, the margin width between the two boundary planes w^T x + b = +1 and w^T x + b = -1 is exactly: Margin = 2 / ||w||.
Maximizing 2 / ||w|| is mathematically equivalent to minimizing the convex quadratic objective (1/2) ||w||^2, guaranteeing a unique global minimum without local optima.
2. Worked Practice Problem: Support Vector Hyperplane Calculation
Use the KKT condition alpha_1 y_1 + alpha_2 y_2 + alpha_3 y_3 = 0 to solve dual multipliers.
Problem: Find the maximum margin hyperplane for 3 support vectors: Positive (+1) at (2, 0) and (0, 2); Negative (-1) at (0, 0).
Step 1: Inspect the geometric midpoint boundary:
The positive points form line x1 + x2 = 2.
The negative point is at (0, 0) where x1 + x2 = 0.
The separating hyperplane lies exactly halfway: x1 + x2 - 1 = 0.
Step 2: Check support vector canonical equations:
At (2, 0): w1(2) + w2(0) + b = +1 => 2 w1 + b = 1
At (0, 2): w1(0) + w2(2) + b = +1 => 2 w2 + b = 1 => w1 = w2
At (0, 0): w1(0) + w2(0) + b = -1 => b = -1
Step 3: Solve for weights:
2 w1 - 1 = 1 => 2 w1 = 2 => w1 = 1, w2 = 1.
Weight vector w = [1, 1]^T, b = -1.
Step 4: Compute Margin width:
||w|| = sqrt(1^2 + 1^2) = sqrt(2).
Margin = 2 / ||w|| = 2 / sqrt(2) = sqrt(2) ≈ 1.414.
Primary Academic References & Exam Sources
Every concept in Notes From a B.Tech Brain is synthesized from authoritative academic texts, seminal peer-reviewed research, verified exam archives, and renowned university lectures.