Random Vectors
Random vectors
Definition 1 (Projection). For a vector x \in \R^n, we can define \pi_i: \R^n \to \R is the projection of an n-length vector onto its i-th component, such that \pi_i(x) = x_i.
Definition 2. The Borel sigma algebra over space \R^n is defined as the smallest sigma algebra generated by the family (\pi_i^{-1}(B_x): x \in \R, i \in [n]) and is denoted by \sB(\R^n). The elements of the Borel sigma algebra are called Borel sets.
Remark 1. By definition of \sB(\R^n), projection \pi_i: \R^n \to \R is a Borel measurable function for all i \in [n]. For a subset A \subseteq \R and projection \pi_i: \R^n \to \R, we can write Thus, for any A \in \sB(\R), we have \pi_i^{-1}(A) \in \sB(\R^n).
Definition 3 (Random vectors). Consider a probability space (\Omega, \sF, P) and a finite n \in \N. A random vector X: \Omega \to \R^n is an \sF-measurable mapping from the sample space to an n-length real-valued vector. That is, for any x \in \R^n, we have
Example 4 (Tuple of indicators). Consider a probability space (\Omega, \sF, P), a finite n \in \N, and events A_1, \dots, A_n \in \sF. We define a mapping X: \Omega \to \set{0,1}^n by X_i(\omega) \triangleq \Ind{A_i}(\omega) for all outcomes \omega \in \Omega. Let x \in \R^n, then we can write A_X(x) = \cap_{i=1}^n\Ind{A_i}^{-1}(-\infty, x_i]. Recall that It follows that the inverse image A_X(x) lies in \sF, and hence X is an \sF-measurable random vector.
Theorem 5. Consider a probability space (\Omega, \sF, P), and a finite n \in \N. A mapping X: \Omega \to \R^n is a random vector if and only if X_i \triangleq \pi_i \circ X: \Omega \to \R are random variables for all i \in [n].
Proof. Proof. We will first show that X: \Omega \to \R^n implies that \pi_i \circ X is a random variable for any i \in [n]. For any i \in [n] and x_i \in \R, we take x = (\infty, \dots, x_i, \dots, \infty). This implies that \pi_i^{-1}(-\infty, x_i] = \R \times \dots \times (-\infty, x_i] \times \dots \times \R \in \sB(\R^n). Further, defining A_{X_i}(x_i) \triangleq X_i^{-1}(-\infty, x_i], we observe from the definition of random vectors that
We will next show that if X_i: \Omega \to \R is a random variable for all i \in [n], then X \triangleq (X_1, \dots, X_n): \Omega \to \R^n is a random vector. For any x \in \R^n, we have A_{X_i}(x_i) = X_i^{-1}(-\infty,x_i] \in \sF for all i \in [n], from the definition of random variables. From the closure of event set under countable intersections, we have ◻
Distribution of random vectors
Definition 6. Consider a probability space (\Omega, \sF, P) and a finite n \in \N. The joint distribution function of a random vector X: \Omega \to \R^n is defined as the mapping F_X: \R^n \to [0,1] such that
Example 7 (Tuple of indicators). Consider a probability space (\Omega, \sF, P), a finite n \in \N, and events A_1, \dots, A_n \in \sF, that define the random vector X \triangleq (\Ind{A_1}, \dots, \Ind{A_n}). For any x \in \R^n, we can define index sets I_0(x) \triangleq \set{i \in [n]: x_i < 0} and I_1(x) \triangleq \set{i \in [n]: x_i \in [0,1)}, and write the joint distribution function for this random vector X as
Definition 8. For a random vector X: \Omega \to \R^n defined on the probability space (\Omega, \sF, P) and i \in [n], the distribution of the ith random variable X_i \triangleq \pi_i\circ X: \Omega \to \R is called the ith marginal distribution, and denoted by F_{X_i}: \Omega \to [0,1].
Example 9 (Tuple of indicators). Consider a probability space (\Omega, \sF, P), a finite n \in \N, and events A_1, \dots, A_n \in \sF, that define the random vector X \triangleq (\Ind{A_1}, \dots, \Ind{A_n}). The ith marginal distribution is given by
Corollary 10 (Marginal distribution). Consider a random vector X: \Omega \to \R^n defined on a probability space (\Omega, \sF, P) with the joint distribution F_X: \R^n \to [0,1]. The ith marginal distribution and can be obtained from the joint distribution of X as
Proof. Proof. For any i \in [n] and x_i \in \R, we have X_i^{-1}(-\infty, x_i] = A_X(x) for x = (\infty, \dots, x_i, \dots, \infty) from [eqn:RandomVector:MarginalEvent]. ◻
Lemma 11 (Properties of the joint distribution function). Consider a random vector X: \Omega \to \R^n defined on the probability space (\Omega,\sF,P). The associated joint distribution function F_X: \R^n \to [0,1] satisfies the following properties.
For x, y \in \R^n such that x_i \le y_i for each i \in [n], we have F_X(x) \le F_X(y).
The function F_X(x) is right continuous at all points x \in \R^n.
The lower limit is \lim_{x_i \to -\infty}F_X(x) = 0, and the upper limit is \lim_{x_i \to \infty, i \in [n]}F_X(x) = 1.
Proof. Proof. Consider a random vector X: \Omega \to \R^n defined on the probability space (\Omega, \sF, P) and any x \in \R^n.
We can verify that A_X(x) = \cap_{i=1}^nA_{X_i}(x_i) \subseteq \cap_{i=1}^nA_{X_i}(y_i) = A(y). The result follows from the monotonicity of probability measure.
The proof is similar to the proof for single random variable.
The event A_X(x) = \emptyset when x_i = -\infty for some i \in [n] and A_X(x) = \Omega when x_i = \infty for all i \in [n], hence the result follow.
◻
Example 12 (Probability of rectangular events). Consider a probability space (\Omega, \sF, P) and a random vector X: \Omega \to \R^2. Consider the points x \le y\in \R^2 and the events The marginal probabilities are given by Writing x = (x_1,x_2) and y = (y_1,y_2), we observe that the end points of the rectangular event B_1\cap B_2 are points x, (y_1,x_2), y, (x_1, y_2). Therefore, we can write this event as Hence, we can write the probability of this rectangular event as
Event space generated by random vectors
Definition 13. Consider a probability space (\Omega, \sF, P) and a finite n \in \N. The event space generated by a random vector X:\Omega \to \R^n is the smallest \sigma-algebra generated by the collection of events (A_X(x): x \in \R^n) and denoted by \sigma(X) \triangleq \sigma(A_X(x):x \in \R^n).
Theorem 14. Consider a probability space (\Omega, \sF, P), a finite n \in \N, a random vector X: \Omega \to \R^n, and its projections X_i \triangleq \pi_i \circ X for all i \in [n]. Then, \sigma(X) = \sigma(X_1, \dots, X_n).
Proof. Proof. Recall that \sigma(X) is generated by the family (A_X(x): x \in \R^n) and \sigma(X_1, \dots, X_n) is generated by the family (A_{X_i}(x_i): x_i \in \R, i \in [n]). We first show that A_{X_i}(x_i) = A_X(x) for x = (\infty, \dots, x_i, \dots, \infty), and hence \sigma(X_1, \dots, X_n) \subseteq \sigma(X). We then show that A_X(x) = \cap_{i=1}^nA_{X_i}(x_i), and hence \sigma(X) \subseteq \sigma(X_1, \dots, X_n). ◻
Example 15 (Tuple of indicators). Consider a probability space (\Omega, \sF, P), a finite n \in \N, and events A_1, \dots, A_n \in \sF, that define the random vector X \triangleq (\Ind{A_1}, \dots, \Ind{A_n}). The \sigma(X) = \sigma(A_i: i \in [n]).
Independence of random variables
Definition 16. A family of collections of events (\sA_i \subseteq \sF: i \in I) is called independent, if for any finite set F \subseteq I and A_i \in \sA_i for all i \in F, we have
Definition 17 (Independent and identically distributed). A random vector X: \Omega \to \R^n defined on the probability space (\Omega, \sF, P) is called independent if The random vector X is called identically distributed if each of its components have the identical marginal distribution, i.e.
Remark 2. Independence of a random vector implies that events (A_{X_i}(x_i): i \in [n]) are independent for any x \in \R^n. Defining families \sA_i \triangleq (A_{X_i}(x): x \in \R) for all i \in [n], we observe that the families (\sA_1, \dots, \sA_n) are mutually independent.
Remark 3. In general, if two collection of events are mutually independent, then the event space generated by them are independent. This can be proved using Dynkin’s \pi-\lambda Theorem.
Theorem 18. For an independent random vector X: \Omega \to \R^n defined on a probability space (\Omega, \sF, P), the event spaces generated by its components (\sigma(X_i): i \in [n]) are independent.
Proof. Proof. For an we define a family of events \sA_i \triangleq (X_i^{-1}(-\infty, x]: x\in \R) for each i \in [n]. From the definition of independence of random vectors, the families (\sA_i \subseteq \sF: i \in [n]) are mutually independent. Since \sigma(\sA_i) = \sigma(X_i), the result follows from the previous remark. ◻
Definition 19 (Independent random vectors). To random vectors X:\Omega \to \R^n and Y: \Omega\to\R^m defined on the same probability space (\Omega, \sF, P) are independent, if the collection of events (A_X(x): x \in \R^n) and (A_Y(y): y \in \R^m) are independent, where A_X(x) \triangleq \cap_{i=1}^nX_i^{-1}(-\infty, x_i] and A_Y(y) \triangleq \cap_{i=1}^mY_i^{-1}(-\infty, y_i].
Example 20 (Independent random vectors). Consider a set of vectors \sX = \set{(0,0,1),(1,0,0)}\subseteq \R^3. Consider two independent coin tosses, such that \Omega = \set{H,T}^2, \sF = \cP(\Omega) and P(\omega) = p^{k_2(\omega)}(1-p)^{2-k_2(\omega)}, where k_2(\omega) = \sum_{i=1}^2\SetIn{\omega_i = H}. We define random vectors
2 &X = (0,0,1) + (1,0,0),& &Y = (0,0,1) + (1,0,0).
We can verify that X,Y: \Omega \to \R^3 are mutually independent random vectors, though we can also check that X_1, X_3 are dependent random variables and so are Y_1, Y_3.
Discrete random vectors
Definition 21 (Discrete random vectors). If a random vector X: \Omega \to \sX_1 \times \dots \times \sX_n \subseteq \R^n takes countable values in \R^n, then it is called a discrete random vector. That is, the range of random vector X is countable, and the random vector is completely specified by the probability mass function
Remark 4. For an independent discrete random vector X: \Omega \to \R^n, we have P_X(x) = \prod_{i=1}^nP_{X_i}(x_i) for each x \in \R^n.
Example 22 (Multiple coin tosses). For a probability space (\Omega, \sF, P), such that \Omega = \set{H,T}^n, \sF = 2^\Omega, P(\omega) = \frac{1}{2^n} for all \omega \in \Omega.
Consider the random vector X: \Omega \to \R such that X_i(\omega) = \SetIn{\omega_i = H} for each i \in [n]. Observe that X is a bijection from the sample space to the set \set{0,1}^n. In particular, X is a discrete random variable.
For any x \in [0,1]^n, we can write N(x) = \sum_{i=1}^n\Ind{[0,1)}(x_i). Further, we can write the joint distribution as We can derive the marginal distribution for i-th component as Therefore, it follows that X is an random vector.
Continuous random vectors
Definition 23 (Joint density function). For jointly continuous random vector X: \Omega \to \R^n with joint distribution function F_X: \R^n \to [0,1], there exists a joint density function f_X: \R^n \to [0,\infty) such that f_X(x) = \frac{d^n}{dx_1\dots dx_n}F_X(x), and
Remark 5. For an independent continuous random vector X: \Omega \to \R^n, we have f_X(x) = \prod_{i=1}^nf_{X_i}(x_i) for all x \in \R^n.
Example 24 (Gaussian random vectors). For a probability space (\Omega, \sF, P), Gaussian random vector is a continuous random vector X: \Omega \to \R^n defined by its density function where the mean vector \mu \in \R^n and the positive definite covariance matrix \Sigma \in \R^{n \times n}. The components of the Gaussian random vector are Gaussian random variables, which are independent when \Sigma is diagonal matrix and are identically distributed when \Sigma = \sigma^2 I.