Chebyshev's Inequality, Law of Large Numbers & Central Limit Theorem
“Markov bound, Chebyshev tail bound, Weak Law of Large Numbers, and Gaussian convergence of sample sums.”
1.Concentration Inequalities & Asymptotic Convergence
§ 3.1.Concentration inequalities allow us to bound probabilities of extreme deviations without knowing the complete underlying distribution.
Markov's Inequality: For non-negative random variable X >= 0 and any constant a > 0: P(X >= a) <= E[X] / a.
Chebyshev's Inequality: Applying Markov's inequality to (X - mu)^2 yields: P(|X - mu| >= epsilon) <= Var(X) / epsilon^2.
Setting epsilon = k sigma gives: P(|X - mu| >= k sigma) <= 1 / k^2.
The Central Limit Theorem (Lindeberg-Levy):
Let X_1, X_2, ..., X_n be independent and identically distributed (i.i.d.) random variables with mean mu and finite variance sigma^2 > 0.
Then regardless of the underlying distribution of X_i (whether Uniform, Exponential, Poisson, or custom), the normalized sample mean converges in distribution to the standard normal distribution:
Z_n = (X_bar - mu) / (sigma / sqrt(n)) ---> N(0, 1) as n -> infinity.
2. Worked Practice Problem: Chebyshev Conservative Bound
Identify the number of standard deviations k: deviation 15 divided by sigma = 5 gives k = 3.
Problem: A random variable X has mean mu = 50 and variance Var(X) = 25. Its probability distribution is completely unknown. Using Chebyshev's inequality, determine the guaranteed minimum probability that X falls strictly between 35 and 65.
Step 1: Compute standard deviation sigma:
sigma = sqrt(Var(X)) = sqrt(25) = 5.
Step 2: Express the interval (35, 65) in terms of deviation from mean mu = 50:
35 < X < 65 <=> -15 < X - 50 < 15 <=> |X - 50| < 15.
Step 3: Relate deviation to standard deviation units k * sigma:
15 = k sigma = k 5 => k = 15 / 5 = 3.
Step 4: Apply Chebyshev's two-sided bounded formula:
P(|X - mu| < k * sigma) >= 1 - (1 / k^2)
Step 5: Substitute k = 3:
P(|X - 50| < 15) >= 1 - (1 / 3^2) = 1 - 1/9 = 8/9 ≈ 0.8889 (88.89%).
Conclusion: Regardless of the shape of the distribution, at least 88.89% (8/9) of the distribution mass is guaranteed to lie between 35 and 65.
Primary Academic References & Exam Sources
Every concept in Notes From a B.Tech Brain is synthesized from authoritative academic texts, seminal peer-reviewed research, verified exam archives, and renowned university lectures.