Joint Distributions, Marginalization & Conditional Expectation
“Double integrals over coupled domains, marginal PDFs, conditional density, and iterated expectation.”
1.Multivariate Random Variables & Marginalization
§ 3.1.Real-world data science problems inherently involve vectors of multiple dependent variables (X_1, X_2, ..., X_d).
Bivariate Probability Density:
For continuous random variables (X, Y), the joint probability density function f(x, y) satisfies:
P((X, Y) in R) = iint_R f(x, y) dx dy.
Marginalization:
To extract the behavior of individual variable X, we integrate out all other dimensions (nuisance variables):
f_X(x) = int_{-infinity}^{+infinity} f(x, y) dy.
Conditional Density & Expectation:
f_{Y|X}(y | x) = f(x, y) / f_X(x).
E[Y | X = x] = int_{-infinity}^{+infinity} y f_{Y|X}(y | x) dy.
2. Worked Practice Problem: Coupled Triangle Domain Joint PDF
When finding f_X(x), integrate y from x to 1; do NOT integrate from 0 to 1.
Problem: Let (X, Y) have joint PDF f(x, y) = c * x for 0 <= x <= y <= 1, and 0 elsewhere. Find the normalization constant c and the marginal expectation E[Y].
Step 1: Set double integral over the triangular region to 1:
Region: 0 <= x <= y and 0 <= y <= 1.
int_0^1 dy int_0^y c * x dx = 1
Step 2: Evaluate inner integral with respect to x:
int_0^y c x dx = c [x^2 / 2]_0^y = c * (y^2 / 2)
Step 3: Evaluate outer integral with respect to y:
int_0^1 (c / 2) y^2 dy = (c / 2) [y^3 / 3]_0^1 = c / 6 = 1 => c = 6.
Joint PDF: f(x, y) = 6x for 0 <= x <= y <= 1.
Step 4: Find marginal PDF f_Y(y):
f_Y(y) = int_0^y 6x dx = 6 * [x^2 / 2]_0^y = 3 y^2 for 0 <= y <= 1.
Step 5: Compute E[Y]:
E[Y] = int_0^1 y f_Y(y) dy = int_0^1 y (3 y^2) dy = 3 * int_0^1 y^3 dy
E[Y] = 3 * [y^4 / 4]_0^1 = 3 / 4 = 0.75.
Conclusion: The normalization constant is c = 6 and the marginal expectation is E[Y] = 3/4.
Primary Academic References & Exam Sources
Every concept in Notes From a B.Tech Brain is synthesized from authoritative academic texts, seminal peer-reviewed research, verified exam archives, and renowned university lectures.