Transformation of random vectors

Functions of random variables

Definition 1. Borel measurable sets on a space \R^n is denoted by \sB(\R^n) and generated by the collection (\pi_i^{-1}(-\infty,x]: x \in \R, i \in [n]). A function g: \R^n\to \R^m is called Borel measurable function, if g^{-1}(B_m) \in \sB(\R^n) for any B_m \in \sB(\R^m).

Proposition 2. Consider a random variable X: \Omega \to \R defined on the probability space (\Omega, \sF, P). Suppose g: \R \to \R is function such that g^{-1}(-\infty, x] \in \sB(\R), then g(X) is a random variable.

Proof. Proof. We represent g(X) by a map Y: \Omega \to \R such that Y(\omega) \triangleq (g \circ X)(\omega) for all outcomes \omega \in \Omega. We further check that for any half open set B_x = (-\infty,x], we have Y^{-1}(B_x) = (X^{-1}\circ g^{-1})(B_x). Since g^{-1}(B_x) \in \sB(\R), it follows that Y^{-1}(B_x) \in \sF by the definition of random variables. ◻

Example 3 (Monotone function of random variables). Let g:\R \to \R be a monotonically increasing function, then g^{-1}(-\infty, x] %= (g^{-1}(-\infty), g^{-1}(x)] \in \sB(\R) for all x \in \R. Consider a random variable X: \Omega \to \R defined on the probability space (\Omega, \sF, P), then Y \triangleq g(X) is a random variable with distribution function Here, g^{-1}(y) is the functional inverse, and not inverse image as we have been seeing typically. We can think g^{-1}(y) = g^{-1}\set{y}, though this inverse image has at most a single element since g is monotonically increasing.

Example 4. Consider a positive random variable X: \Omega \to \R_+ defined on a probability space (\Omega,\sF,P). Let g: \R_+ \to \R_+ be such that g(x) = e^{-\theta x} for all x \in \R_+ and some \theta > 0. Then, g is monotonically decreasing in X and x = g^{-1}(y) = -\frac{1}{\theta}\ln y. This implies that g^{-1}(-\infty, y] = [-\frac{1}{\theta}\ln y, \infty) \in \sB(\R_+) for all y \in \R_+. Thus g is a measurable function, and Y = g(X) is a random variable.

Proposition 5 (Independence of function of random variables). Let g:\R \to \R and h: \R \to \R be functions such that g^{-1}(-\infty, x] and h^{-1}(-\infty, x] are Borel sets for all x \in \R. Consider independent random variables X and Y defined on the probability space (\Omega, \sF, P), then g(X) and h(Y) are independent random variables.

Proof. Proof. For any u,v \in \R, we can define inverse images A_g(u) \triangleq g^{-1}(-\infty,u] and A_h(v) \triangleq h^{-1}(\infty, v]. Since g,h are Borel measurable, we have A_g(u), A_h(v) \in \sB(\R). We can write the following outcome set equality for the joint event Since X and Y are independent random variables, it follows that X^{-1}(A_g(u)) and Y^{-1}(A_h(v)) are independent events, and the result follows. ◻

Function of random vectors

Proposition 6. Consider a random vector X: \Omega \to \R^n defined on the probability space (\Omega, \sF, P), and a Borel measurable function g: \R^n \to \R^m such that A_g(y) \triangleq \cap_{j=1}^m\set{x \in \R^n: g_j(x) \le y_j} \in \sB(\R^n) for all y \in \R^m. Then, g(X): \Omega \to \R^m is a random vector. The joint distribution function F_Y: \R^m \to [0,1] for the vector Y\triangleq g(X) is given by

Example 7 (Sum of random variables). For a random vector X: \Omega \to \R^n defined on a probability space (\Omega, \sF, P). Define an addition function +: \R^n \to \R such that +(x) = \sum_{i=1}^nx_i for any x \in \R^n. We can verify that + is a Borel measurable function and hence Y = +(X) = \sum_{i=1}^nX_i is a random variable. When n=2 and X is a continuous random vector with density f_X: \R^2 \to \R_+, we can write By applying a change of variable (x_1, t)= (x_1, x_1 + x_2) and changing the order of integration, we see that When Y is a continuous random vector, we can write When X: \Omega \to \R^2 is an independent vector, then f_X(x) = f_{X_1}(x_1)f_X(x_2) for all x\in\R^2. Therefore, the density of the sum X_1+X_2 is given by where \ast: \R^{\R} \times \R^{\R} \to \R^{\R} is the convolution operator.

Theorem 8. For a continuous random vector X:\Omega \to \R^m defined on the probability space (\Omega, \sF, P) with density f_X: \R^m \to \R_+ and an injective and smooth Borel measurable function g: \R^m \to \R^m, such that Y=g(X) is a continuous random vector. Then the density of random vector Y is given by where x=g^{-1}(y) and J(y)= (J_{ij}(y) \triangleq \frac{\partial y_j}{\partial x_i}: i,j\in [m]) is the Jacobian matrix.

Proof. Proof. For an injective map g: \R^m \to \R^m we have \set{x} = g^{-1}\set{y} for any y \in g(\R^m). Further, since g is smooth, we have dy = J(y)dx + o(\abs{dx}), and thus Defining set dB(y) \triangleq \set{w \in \R^m: y_j \le w_j \le y_j+dy_j}, we observe that for any continuous random vector Y: \Omega \to \R^m, we have We get the result by combining [eqn:Jacobian] and [eqn:density]. ◻

Example 9 (Sum of random variables). Suppose that X: \Omega \to \R^2 is a continuous random vector and Y_1 = X_1 + X_2. Let us compute f_{Y_1}(y_1) using the above theorem. Let us define a random vector Y: \Omega \to \R^2 such that Y = (X_1+X_2, X_2) so that \abs{J(y)} = 1. This implies, f_{Y}(y) = f_{X}(x). Thus, we may compute the marginal density of Y_1 as, If X is an independent random vector, then where * represents convolution.