Combinatorics, Probability Axioms & Conditional Independence
“Permutations, combinations, Kolmogorov axioms, and independent vs mutually exclusive events.”
1.Sample Spaces, Set Theory & Kolmogorov's Axioms
§ 1.1.Probability theory mathematically formalizes uncertainty over a sample space Omega of all possible outcomes of a random experiment.
Kolmogorov's Three Axioms:
1. Non-negativity: For every event A in the event space F, P(A) >= 0.
2. Unit Measure: The probability of the entire sample space is unity: P(Omega) = 1.
3. Countable Additivity: For any countable sequence of pairwise disjoint events A_1, A_2, ... where A_i ∩ A_j = empty for i != j:
P( Union_{i=1}^infinity A_i ) = Sum_{i=1}^infinity P(A_i).
Independent vs Mutually Exclusive Events:
Two events A and B are independent if knowledge that B occurred does not alter the likelihood of A: P(A | B) = P(A) <=> P(A ∩ B) = P(A) P(B). By contrast, A and B are mutually exclusive (disjoint) if they cannot occur simultaneously: A ∩ B = empty => P(A ∩ B) = 0.
If two events A and B each have non-zero probabilities, they CANNOT be both mutually exclusive and independent. If they are mutually exclusive, P(A ∩ B) = 0, but independence requires P(A ∩ B) = P(A)P(B) > 0. This conceptual contradiction is tested frequently in GATE DA.
2. Worked Practice Problem: Combinatorial Feature Selection Probability
Use the complement rule: P(At least one collinear) = 1 - P(Zero collinear selected).
Problem: An automated machine learning feature selector randomly picks a subset of 3 features from an available set of 8 candidate features. Unknown to the system, exactly 2 of the 8 features are highly collinear. Find the probability that the selected subset contains at least one of the collinear features.
Step 1: Compute the total number of equally likely ways to select 3 features from 8:
Total outcomes |S| = C(8, 3) = 8! / [ 3! (8 - 3)! ] = (8 7 6) / (3 2 * 1) = 56.
Step 2: Compute the number of ways to pick 3 features containing NO collinear features:
There are 8 - 2 = 6 non-collinear features.
Number of ways to choose 3 from these 6: C(6, 3) = (6 5 4) / (3 2 1) = 20.
Step 3: Calculate the probability of selecting zero collinear features:
P(Zero collinear) = C(6, 3) / C(8, 3) = 20 / 56 = 5 / 14 ≈ 0.3571.
Step 4: Apply the complement rule for 'at least one':
P(At least one collinear) = 1 - P(Zero collinear) = 1 - (5 / 14) = 9 / 14 ≈ 0.6429 (64.29%).
Conclusion: The probability that the subset contains at least one collinear feature is exactly 9/14.
Primary Academic References & Exam Sources
Every concept in Notes From a B.Tech Brain is synthesized from authoritative academic texts, seminal peer-reviewed research, verified exam archives, and renowned university lectures.