Random Variable
Random Variable
Definition 1 (Random variable). Consider a probability space \((\Omega, \sF, P)\). A random variable \(X: \Omega \to \R\) is a real-valued function from the sample space to real numbers, such that for each \(x \in \R\) the event We say that the random variable \(X\) is \(\sF\)-measurable.
Remark 1. Recall that the set \(A_X(x)\) is always a subset of sample space \(\Omega\) for any mapping \(X:\Omega\to\R\), and \(A_X(x)\in\sF\) is an event when \(X\) is a random variable.
Example 2 (Constant function). Consider a mapping \(X:\Omega\to \set{c} \subseteq \R\) defined on an arbitrary probability space \((\Omega, \sF, P)\), such that \(X(\omega) = c\) for all outcomes \(\omega \in \Omega\). We observe that That is \(A_X(x) \in \sF\) for all event spaces, and hence \(X\) is a random variable and measurable for all event spaces.
Example 3 (Indicator function). For an arbitrary probability space \((\Omega,\sF, P)\) and an event \(A \in \sF\), consider the indicator function \(\Ind{A}:\Omega \to [0,1]\). Let \(x \in \R\), and \(B_x = (-\infty, x]\), then it follows that That is, \(A_X(x) \in \sF\) for all \(x \in \R\), and hence the indicator function \(\Ind{A}\) is a random variable.
Remark 2. Since any outcome \(\omega \in \Omega\) is random, so is the real value \(X(\omega)\).
Remark 3. Probability is defined only for events and not for random variables. The events of interest for random variables are the lower level sets \(A_X(x) = \set{\omega: X(\omega) \le x} = X^{-1}(B_x)\) for any real \(x\).
Remark 4. Consider a probability space \((\Omega,\sF,P)\) and a random variable \(X:\Omega \to \R\) that is \(\sG\) measurable for \(\sG \subseteq \sF\). If \(\sG \subseteq \sH\), then \(X\) is also \(\sH\) measurable.
Distribution function for a random variable
Definition 4. For an \(\sF\) measurable random variable \(X:\Omega\to\R\) defined on the probability space \((\Omega,\sF,P)\), we can associate a distribution function (CDF) \(F_X: \R \to [0,1]\) such that for all \(x \in \R\),
Example 5 (Constant random variable). Let \(X:\Omega \to \set{c} \subseteq \R\) be a constant random variable defined on the probability space \((\Omega, \sF, P)\). The distribution function is a right-continuous step function at \(c\) with step-value unity. That is, \(F_X(x) = \Ind{[c,\infty)}(x)\). We observe that \(P(\set{X = c}) = 1\).
Example 6 (Indicator random variable). For an indicator random variable \(\Ind{A}:\Omega \to \set{0,1}\) defined on a probability space \((\Omega, \sF, P)\) and an event \(A \in \sF\), we have
Lemma 7 (Properties of distribution function). The distribution function \(F_X\) for any random variable \(X\) satisfies the following properties.
The distribution function is monotonically non-decreasing in \(x \in \R\).
The distribution function is right-continuous at all points \(x \in \R\).
The upper limit is \(\lim_{x \to \infty}F_X(x) = 1\) and the lower limit is \(\lim_{x \to -\infty}F_X(x) = 0\).
Proof. Proof. Let \(X\) be a random variable defined on the probability space \((\Omega, \sF, P)\).
Let \(x_1, x_2 \in \R\) such that \(x_1 \le x_2\). Then for any \(\omega \in A_{x_1}\), we have \(X(\omega) \le x_1 \le x_2\), and it follows that \(\omega \in A_{x_2}\). This implies that \(A_{x_1} \subseteq A_{x_2}\). The result follows from the monotonicity of the probability.
For any \(x \in \R\), consider any monotonically decreasing sequence \(x\in\R^\N\) such that \(\lim_n x_n = x_0\). It follows that the sequence of events \(\big(A_{x_n} = X^{-1}(-\infty, x_n] \in \sF: n \in \N\big)\), is monotonically decreasing and hence \(\lim_{n \in \N}A_{x_n} = \cap_{n \in \N}A_{x_n} = A_{x_0}\). The right-continuity then follows from the continuity of probability, since
Consider a monotonically increasing sequence \(x\in\R^\N\) such that \(\lim_{n}x_n = \infty\), then \((A_{x_n} \in \sF: n \in \N)\) is a monotonically increasing sequence of sets and \(\lim_nA_{x_n} = \cup_{n \in \N}A_{x_n} = \Omega\). From the continuity of probability, it follows that Similarly, we can take a monotonically decreasing sequence \(x\in \R^\N\) such that \(\lim_{n}x_n = -\infty\), then \((A_{x_n} \in \sF: n \in \N)\) is a monotonically decreasing sequence of sets and \(\lim_nA_{x_n} = \cap_{n \in \N}A_{x_n} = \emptyset\). From the continuity of probability, it follows that \(\lim_{x_n \to -\infty}F_X(x_n) = 0\).
◻
Remark 5. If two reals \(x_1 < x_2\) then \(F_X(x_1) \le F_X(x_2)\) with equality if and only if \(P\set{(x_1 < X \le x_2}) = 0\). This follows from the fact that \(A_{x_2} = A_{x_1}\cup X^{-1}(x_1, x_2]\).
Event space generated by a random variable
Definition 8 (Event space generated by a random variable). Let \(X: \Omega \to \R\) be an \(\sF\) measurable random variable defined on the probability space \((\Omega, \sF, P)\). The smallest event space generated by the events \(A_X(x) = X^{-1}(B_x) = X^{-1}(-\infty, x]\) for \(x \in \R\) is called the event space generated by this random variable \(X\), and denoted by \(\sigma(X) \triangleq \sigma(\set{A_X(x): x \in \R})\).
Remark 6. The event space generated by a random variable is the collection of the inverse of Borel sets, i.e. \(\sigma(X) = \set{X^{-1}(B): B \in \sB(\R)}\). This follows from the fact that \(A_X(x) = X^{-1}(B_x)\) and the inverse map respects countable set operations such as unions, complements, and intersections. That is, if \(B \in \sB(\R) = \sigma(\set{B_x:x\in \R})\), then \(X^{-1}(B) \in \sigma(\set{A_X(x): x \in \R})\). Similarly, if \(A \in \sigma(X) = \sigma(\set{A_X(x): x \in \R})\), then \(A = X^{-1}(B)\) for some \(B \in \sigma(\set{B_x: x \in \R})\).
Example 9 (Constant random variable). Let \(X:\Omega \to \set{c} \subseteq \R\) be a constant random variable defined on the probability space \((\Omega,\sF,P)\). Then the smallest event space generated by this random variable is \(\sigma(X) = \set{\emptyset, \Omega}\).
Example 10 (Indicator random variable). Let \(\Ind{A}\) be an indicator random variable defined on the probability space \((\Omega,\sF,P)\) and event \(A \in \sF\), then the smallest event space generated by this random variable is \(\sigma(X) = \sigma(\set{\emptyset, A^c, \Omega}) = \set{\emptyset, A, A^c, \Omega}\).
Discrete random variables
Definition 11 (Discrete random variables). If a random variable \(X: \Omega \to \sX \subseteq \R\) takes countable values on real-line, then it is called a discrete random variable. That is, the range of random variable \(\sX\) is countable, and the random variable is completely specified by the probability mass function
Example 12 (Bernoulli random variable). For the probability space \((\Omega, \sF, P)\), the Bernoulli random variable is a mapping \(X: \Omega \to \set{0,1}\) and \(P_X(1) = p\). We observe that Bernoulli random variable is an indicator for the event \(A \triangleq X^{-1}\set{1}\), and \(P(A) = p\). Therefore, the distribution function \(F_X\) is given by
Lemma 13. Any discrete random variable is a linear combination of indicator function over a partition of the sample space.
Proof. Proof. For a discrete random variable \(X: \Omega \to \sX \subset \R\) on a probability space \((\Omega, \sF, P)\), the range \(\sX\) is countable, and we can define events \(E_x \triangleq \set{\omega \in \Omega: X(\omega) = x} \in \sF\) for each \(x \in \sX\). Then the mutually disjoint sequence of events \((E_x \in \sF: x \in \sX)\) partitions the sample space \(\Omega\). We can write ◻
Definition 14. Any discrete random variable \(X:\Omega \to \sX \subseteq\R\) defined over a probability space \((\Omega,\sF,P)\), with finite range is called a simple random variable.
Example 15 (Simple random variables). Let \(X\) be a simple random variable, then \(X = \sum_{x \in \sX}x\Ind{A_X(x)}\) where \((A_X(x) = X^{-1}\set{x} \in \sF: x \in \sX)\) is a finite partition of the sample space \(\Omega\). Without loss of generality, we can denote \(\sX = \set{x_1, \dots, x_n}\) where \(x_1 \le \dots \le x_n\). Then, Then the smallest event space generated by the simple random variable \(X\) is \(\set{\cup_{x \in S}A_X(x): S \subseteq \sX}\).
Continuous random variables
Definition 16. For a continuous random variable \(X\), there exists density function \(f_X: \R \to [0, \infty)\) such that
Example 17 (Gaussian random variable). For a probability space \((\Omega, \sF, P)\), Gaussian random variable is a continuous random variable \(X: \Omega \to \R\) defined by its density function