Random Variables and Discrete Random Variables

# Random Variables and Discrete Random Variables ## Objectifs et prérequis This chapter introduces the concept of random variables, focusing on discrete random variables and their associated probability distributions. Students will learn to model uncertain outcomes numerically, compute key statistical measures such as expectation and variance, and analyze the behavior of sums of independent random variables. The chapter also covers probability generating functions as a tool for deriving distributions of sums. **Prerequisites**: - Basic probability theory (sample spaces, events, axioms of probability) - Combinatorics (permutations, combinations) - Algebraic manipulation and summation notation - Familiarity with functions and sequences --- ## I. Understanding random variables A **random variable** is a function that assigns a real number to each outcome in a sample space. It allows us to quantify uncertainty by mapping random phenomena to numerical values. > **Definition**: A random variable $X$ is a function $X: \Omega \to \mathbb{R}$, where $\Omega$ is the sample space of a random experiment. Random variables are classified as **discrete** or **continuous** based on the nature of their possible values. A discrete random variable takes on a countable number of distinct values (e.g., integers), while a continuous random variable can take any value in an interval. > **Example**: Tossing a fair coin twice. Let $X$ be the number of heads. > Sample space: $\Omega = \{HH, HT, TH, TT\}$ > Then: > $X(HH) = 2$, $X(HT) = 1$, $X(TH) = 1$, $X(TT) = 0$ > So $X$ takes values in $\{0, 1, 2\}$ — a discrete random variable. > **To remember**: A random variable is not a variable in the algebraic sense; it is a function whose value depends on the outcome of a random process. ### Exercises **Exercise 1**: A die is rolled. Let $Y$ be the outcome if the number is even, and $Y = 0$ if odd. List the possible values of $Y$ and determine if $Y$ is discrete. **Solution**: - Even outcomes: 2, 4, 6 → $Y = 2, 4, 6$ - Odd outcomes: 1, 3, 5 → $Y = 0$ - Possible values: $\{0, 2, 4, 6\}$ — finite and countable → discrete. **Exercise 2**: Two coins are tossed. Define $Z$ as the number of tails minus the number of heads. Is $Z$ discrete? List its values. **Solution**: - HH: $Z = 0 - 2 = -2$ - HT: $Z = 1 - 1 = 0$ - TH: $Z = 1 - 1 = 0$ - TT: $Z = 2 - 0 = 2$ - Values: $\{-2, 0, 2\}$ — discrete. --- ## II. Discrete random variables A **discrete random variable** is one that can take on only a finite or countably infinite number of distinct values. > **Definition**: A random variable $X$ is discrete if its range $R_X = \{x_1, x_2, \dots\}$ is countable (finite or countably infinite). Discrete random variables are fully described by their **probability mass function (PMF)**, which gives the probability that $X$ takes each specific value. > **Example**: Number of customers arriving at a shop in an hour, modeled as a Poisson variable (countably infinite support). > **Example**: Score on a 10-question true/false quiz — values $0$ to $10$ — finite discrete. > **To remember**: Discrete random variables arise from counting processes; their values are isolated points on the real line. ### Exercises **Exercise 3**: A box contains 3 red and 2 blue balls. Two balls are drawn without replacement. Let $X$ be the number of red balls drawn. Show $X$ is discrete and find its range. **Solution**: - Possible outcomes: RR, RB, BR, BB - $X(RR)=2$, $X(RB)=1$, $X(BR)=1$, $X(BB)=0$ - Range: $\{0,1,2\}$ — finite → discrete. **Exercise 4**: Let $X$ be the number of trials until the first success in repeated Bernoulli trials with success probability $p$. Is $X$ discrete? **Solution**: Yes — $X$ can take values $1,2,3,\dots$ — countably infinite → discrete (geometric distribution). --- ## III. Probability distributions The **probability distribution** of a discrete random variable $X$ specifies the probability associated with each possible value of $X$. It can be presented as a table, formula, or graph. > **Definition**: The probability distribution of a discrete random variable $X$ is the set of pairs $(x_i, P(X = x_i))$ for all $x_i$ in the range of $X$, satisfying: > 1. $P(X = x_i) \geq 0$ for all $i$ > 2. $\sum_{i} P(X = x_i) = 1$ > **Example**: Fair die roll. Let $X$ be the outcome. > $P(X=k) = \frac{1}{6}$ for $k=1,2,3,4,5,6$ > Sum: $6 \times \frac{1}{6} = 1$ — valid distribution. > **To remember**: A valid probability distribution must assign non-negative probabilities that sum to exactly 1. ### Exercises **Exercise 5**: Is the following a valid probability distribution for $X$? $$ \begin{array}{c|c} x & P(X=x) \\ \hline 0 & 0.2 \\ 1 & 0.5 \\ 2 & 0.4 \\ \end{array} $$ **Solution**: Sum = $0.2 + 0.5 + 0.4 = 1.1 > 1$ → invalid. **Exercise 6**: Find $k$ so that the following is a valid probability distribution: $$ P(X=x) = \begin{cases} kx & \text{for } x = 1,2,3 \\ 0 & \text{otherwise} \end{cases} $$ **Solution**: $\sum_{x=1}^3 kx = k(1+2+3) = 6k = 1 \Rightarrow k = \frac{1}{6}$ --- ## IV. Probability density or mass function For discrete random variables, we use the **probability mass function (PMF)**, not a density function (which applies to continuous variables). > **Definition**: The probability mass function (PMF) of a discrete random variable $X$ is the function $p_X(x) = P(X = x)$ for all $x \in \mathbb{R}$, with $p_X(x) = 0$ for $x$ not in the range of $X$. The PMF must satisfy: 1. $p_X(x) \geq 0$ for all $x$ 2. $\sum_{x \in \mathbb{R}} p_X(x) = 1$ > **Example**: Bernoulli distribution with parameter $p$: > $p_X(0) = 1-p$, $p_X(1) = p$, $p_X(x)=0$ otherwise > Sum: $(1-p) + p = 1$ — valid. > **Example**: Binomial distribution: $X \sim \text{Bin}(n,p)$ > $p_X(k) = \binom{n}{k} p^k (1-p)^{n-k}$ for $k=0,1,\dots,n$ > **To remember**: PMF gives probability at exact points; for discrete variables, $P(X=x) = p_X(x) > 0$ is possible. ### Exercises **Exercise 7**: Let $X$ have PMF: $p_X(x) = \frac{c}{x^2}$ for $x = 1,2,3,\dots$, and 0 otherwise. Find $c$. **Solution**: $\sum_{x=1}^\infty \frac{c}{x^2} = c \sum_{x=1}^\infty \frac{1}{x^2} = c \cdot \frac{\pi^2}{6} = 1 \Rightarrow c = \frac{6}{\pi^2}$ **Exercise 8**: A random variable $X$ has PMF: $p_X(0)=0.1$, $p_X(1)=0.3$, $p_X(2)=0.4$, $p_X(3)=0.2$. Verify it is valid and compute $P(X \geq 2)$. **Solution**: Sum: $0.1+0.3+0.4+0.2=1.0$ → valid. $P(X \geq 2) = p_X(2)+p_X(3)=0.4+0.2=0.6$ --- ## V. Expectation E(X) or mean / Standard deviation The **expectation** (or expected value) of a discrete random variable is the long-run average value of repetitions of the experiment it represents. > **Definition**: The expectation of a discrete random variable $X$ is > $$ > \mathbb{E}[X] = \sum_{x} x \cdot p_X(x) > $$ > where the sum is over all possible values of $X$. The **variance** measures the spread of $X$ around its mean: > **Definition**: > $$ > \text{Var}(X) = \mathbb{E}[(X - \mathbb{E}[X])^2] = \sum_{x} (x - \mu)^2 p_X(x) > $$ > where $\mu = \mathbb{E}[X]$. > An equivalent formula is: > $$ > \text{Var}(X) = \mathbb{E}[X^2] - (\mathbb{E}[X])^2 > $$ The **standard deviation** is the square root of the variance: > $$ > \sigma_X = \sqrt{\text{Var}(X)} > $$ > **Example**: Fair die. $\mathbb{E}[X] = \sum_{k=1}^6 k \cdot \frac{1}{6} = \frac{21}{6} = 3.5$ > $\mathbb{E}[X^2] = \sum_{k=1}^6 k^2 \cdot \frac{1}{6} = \frac{91}{6} \approx 15.1667$ > $\text{Var}(X) = \frac{91}{6} - (3.5)^2 = \frac{91}{6} - \frac{49}{4} = \frac{182 - 147}{12} = \frac{35}{12} \approx 2.9167$ > $\sigma_X = \sqrt{\frac{35}{12}} \approx 1.708$ > **To remember**: Expectation is a weighted average; variance measures dispersion; standard deviation is in the same units as $X$. ### Exercises **Exercise 9**: Let $X$ have PMF: $p_X(0)=0.2$, $p_X(1)=0.5$, $p_X(2)=0.3$. Compute $\mathbb{E}[X]$, $\text{Var}(X)$, and $\sigma_X$. **Solution**: $\mathbb{E}[X] = 0\cdot0.2 + 1\cdot0.5 + 2\cdot0.3 = 0 + 0.5 + 0.6 = 1.1$ $\mathbb{E}[X^2] = 0^2\cdot0.2 + 1^2\cdot0.5 + 2^2\cdot0.3 = 0 + 0.5 + 1.2 = 1.7$ $\text{Var}(X) = 1.7 - (1.1)^2 = 1.7 - 1.21 = 0.49$ $\sigma_X = \sqrt{0.49} = 0.7$ **Exercise 10**: For a Bernoulli($p$) variable, show $\mathbb{E}[X]=p$ and $\text{Var}(X)=p(1-p)$. **Solution**: $\mathbb{E}[X] = 0\cdot(1-p) + 1\cdot p = p$ $\mathbb{E}[X^2] = 0^2\cdot(1-p) + 1^2\cdot p = p$ $\text{Var}(X) = p - p^2 = p(1-p)$ --- ## VI. The cumulative distribution function The **cumulative distribution function (CDF)** gives the probability that $X$ is less than or equal to a given value. > **Definition**: The CDF of a discrete random variable $X$ is > $$ > F_X(x) = P(X \leq x) = \sum_{t \leq x} p_X(t) > $$ > for all $x \in \mathbb{R}$. The CDF is a non-decreasing, right-continuous function that satisfies: - $\lim_{x \to -\infty} F_X(x) = 0$ - $\lim_{x \to \infty} F_X(x) = 1$ - $F_X(x)$ is constant between jumps at the values of $X$. > **Example**: $X \sim \text{Bernoulli}(0.4)$ > $F_X(x) = > \begin{cases} > 0 & x < 0 \\ > 0.6 & 0 \leq x < 1 \\ > 1 & x \geq 1 > \end{cases} > $ > **To remember**: The CDF accumulates probabilities from the left; jumps occur at the values of $X$ equal to the PMF values. ### Exercises **Exercise 11**: Let $X$ have PMF: $p_X(1)=0.2$, $p_X(2)=0.5$, $p_X(3)=0.3$. Find $F_X(x)$ for all $x$. **Solution**: - $x < 1$: $F_X(x)=0$ - $1 \leq x < 2$: $F_X(x)=p_X(1)=0.2$ - $2 \leq x < 3$: $F_X(x)=0.2+0.5=0.7$ - $x \geq 3$: $F_X(x)=0.2+0.5+0.3=1.0$ **Exercise 12**: Use the CDF to find $P(1 < X \leq 3)$ from Exercise 11. **Solution**: $P(1 < X \leq 3) = F_X(3) - F_X(1) = 1.0 - 0.2 = 0.8$ (Alternatively: $p_X(2)+p_X(3)=0.5+0.3=0.8$) --- ## VII. The Mode and the Median The **mode** is the value of $X$ that occurs most frequently; the **median** is the middle value of the distribution. > **Definition**: The **mode** of a discrete random variable $X$ is the value $x$ that maximizes $p_X(x)$. > If multiple values have the same maximum probability, the distribution is multimodal. > **Definition**: The **median** $m$ of $X$ is any number such that > $$ > P(X \leq m) \geq \frac{1}{2} \quad \text{and} \quad P(X \geq m) \geq \frac{1}{2} > $$ > For discrete variables, it is often the smallest $x$ such that $F_X(x) \geq 0.5$. > **Example**: PMF: $p_X(0)=0.1$, $p_X(1)=0.6$, $p_X(2)=0.2$, $p_X(3)=0.1$ > Mode: $x=1$ (highest probability) > CDF: $F_X(0)=0.1$, $F_X(1)=0.7$, $F_X(2)=0.9$, $F_X(3)=1.0$ > Median: smallest $x$ with $F_X(x) \geq 0.5$ → $x=1$ > **To remember**: Mode = peak of PMF; median = 50th percentile from CDF. ### Exercises **Exercise 13**: Find the mode and median of $X$ with PMF: $p_X(1)=0.3$, $p_X(2)=0.4$, $p_X(3)=0.2$, $p_X(4)=0.1$. **Solution**: - Mode: $x=2$ (max PMF = 0.4) - CDF: $F_X(1)=0.3$, $F_X(2)=0.7$, $F_X(3)=0.9$, $F_X(4)=1.0$ - Median: smallest $x$ with $F_X(x) \geq 0.5$ → $x=2$ **Exercise 14**: For a symmetric distribution about $\mu$, how do mean, median, and mode relate? **Solution**: In a symmetric unimodal discrete distribution, mean = median = mode. --- ## VIII. The distribution of $X_1 + X_2$ When two independent discrete random variables are added, the distribution of their sum is found via convolution. > **Definition**: If $X_1$ and $X_2$ are independent discrete random variables with PMFs $p_{X_1}$ and $p_{X_2}$, then the PMF of $Y = X_1 + X_2$ is > $$ > p_Y(y) = \sum_{x} p_{X_1}(x) \cdot p_{X_2}(y - x) > $$ > where the sum is over all $x$ such that $p_{X_1}(x) > 0$ and $p_{X_2}(y-x) > 0$. > **Example**: Let $X_1, X_2 \sim \text{Bernoulli}(0.5)$, independent. Find distribution of $Y = X_1 + X_2$. > Possible values: $0,1,2$ > - $P(Y=0) = P(X_1=0,X_2=0) = 0.5 \times 0.5 = 0.25$ > - $P(Y=1) = P(0,1)+P(1,0) = 0.25+0.25=0.5$ > - $P(Y=2) = P(1,1)=0.25$ > So $Y \sim \text{Bin}(2,0.5)$ > **To remember**: The sum of independent random variables uses convolution; for independent Bernoulli trials, the sum is binomial. ### Exercises **Exercise 15**: Let $X_1 \sim \text{Bin}(2,0.5)$, $X_2 \sim \text{Bin}(1,0.5)$, independent. Find $P(Y=2)$ where $Y=X_1+X_2$. **Solution**: $X_1$: $P(0)=0.25$, $P(1)=0.5$, $P(2)=0.25$ $X_2$: $P(0)=0.5$, $P(1)=0.5$ $Y=2$ when: - $X_1=2, X_2=0$: $0.25 \times 0.5 = 0.125$ - $X_1=1, X_2=1$: $0.5 \times 0.5 = 0.25$ - $X_1=0, X_2=2$: impossible Total: $0.125 + 0.25 = 0.375$ **Exercise 16**: Two fair dice are rolled. Let $X$ be the sum. Use convolution to find $P(X=7)$. **Solution**: Each die: $P(k)=\frac{1}{6}$ for $k=1,\dots,6$ $P(X=7) = \sum_{i=1}^6 P(X_1=i) P(X_2=7-i)$ Valid $i$: 1 to 6 (since $7-i$ between 1 and 6) Each term: $\frac{1}{6} \cdot \frac{1}{6} = \frac{1}{36}$ 6 terms → $6 \times \frac{1}{36} = \frac{1}{6}$ --- ## IX. Probability generating function The **probability generating function (PGF)** is a powerful tool for analyzing discrete random variables, especially for sums and moments. > **Definition**: The probability generating function of a discrete random variable $X$ taking non-negative integer values is > $$ > G_X(t) = \mathbb{E}[t^X] = \sum_{k=0}^\infty p_X(k) t^k > $$ > defined for $|t| \leq 1$ (or where the series converges). Key properties: - $G_X(1) = 1$ (since $\sum p_X(k) = 1$) - $G_X'(t) = \sum_{k=1}^\infty k p_X(k) t^{k-1}$ → $G_X'(1) = \mathbb{E}[X]$ - $G_X''(t) = \sum_{k=2}^\infty k(k-1) p_X(k) t^{k-2}$ → $G_X''(1) = \mathbb{E}[X(X-1)]$ - Then: $\text{Var}(X) = G_X''(1) + G_X'(1) - [G_X'(1)]^2$ For independent $X_1, X_2$: > $$ > G_{X_1+X_2}(t) = G_{X_1}(t) \cdot G_{X_2}(t) > $$ > **Example**: $X \sim \text{Bernoulli}(p)$ > $G_X(t) = p_X(0) t^0 + p_X(1) t^1 = (1-p) + p t$ > $G_X'(t) = p$ → $G_X'(1)=p = \mathbb{E}[X]$ > $G_X''(t)=0$ → $\mathbb{E}[X(X-1)]=0$ > $\text{Var}(X) = 0 + p - p^2 = p(1-p)$ — correct. > **Example**: $X \sim \text{Bin}(n,p)$ > $G_X(t) = \sum_{k=0}^n \binom{n}{k} p^k (1-p)^{n-k} t^k = [(1-p) + p t]^n$ > (Binomial theorem) > **To remember**: PGF turns convolution into product; derivatives at $t=1$ give moments. ### Exercises **Exercise 17**: Find the PGF of $X \sim \text{Poisson}(\lambda)$ and use it to find $\mathbb{E}[X]$ and $\text{Var}(X)$. **Solution**: $p_X(k) = e^{-\lambda} \frac{\lambda^k}{k!}$ for $k=0,1,2,\dots$ $G_X(t) = \sum_{k=0}^\infty e^{-\lambda} \frac{\lambda^k}{k!} t^k = e^{-\lambda} \sum_{k=0}^\infty \frac{(\lambda t)^k}{k!} = e^{-\lambda} e^{\lambda t} = e^{\lambda(t-1)}$ $G_X'(t) = \lambda e^{\lambda(t-1)}$ → $G_X'(1) = \lambda = \mathbb{E}[X]$ $G_X''(t) = \lambda^2 e^{\lambda(t-1)}$ → $G_X''(1) = \lambda^2 = \mathbb{E}[X(X-1)]$ $\text{Var}(X) = \lambda^2 + \lambda - \lambda^2 = \lambda$ **Exercise 18**: Let $X_1 \sim \text{Bin}(2,0.5)$, $X_2 \sim \text{Bin}(3,0.5)$, independent. Find the PGF of $Y=X_1+X_2$ and hence $\mathbb{E}[Y]$. **Solution**: $G_{X_1}(t) = (0.5 + 0.5t)^2$ $G_{X_2}(t) = (0.5 + 0.5t)^3$ $G_Y(t) = (0.5+0.5t)^2 \cdot (0.5+0.5t)^3 = (0.5+0.5t)^5$ So $Y \sim \text{Bin}(5,0.5)$ $\mathbb{E}[Y] = 5 \times 0.5 = 2.5$ (Or: $G_Y'(t) = 5 \cdot 0.5 (0.5+0.5t)^4$ → $G_Y'(1)=2.5$) --- ## Synthèse du chapitre - A random variable maps outcomes to real numbers; discrete if its range is countable. - The probability mass function (PMF) gives $P(X=x)$ and must sum to 1. - Expectation $\mathbb{E}[X] = \sum x p_X(x)$ is the mean; variance $\text{Var}(X) = \mathbb{E}[X^2] - (\mathbb{E}[X])^2$ measures spread. - The cumulative distribution function (CDF) $F_X(x) = P(X \leq x)$ is non-decreasing and right-continuous. - The mode maximizes the PMF; the median satisfies $P(X \leq m) \geq 0.5$ and $P(X \geq m) \geq 0.5$. - For independent $X_1, X_2$, the PMF of $X_1+X_2$ is the convolution of their PMFs. - The probability generating function $G_X(t) = \mathbb{E}[t^X]$ simplifies moment and sum calculations: $G_X'(1)=\mathbb{E}[X]$, $G_X''(1)=\mathbb{E}[X(X-1)]$, and $G_{X_1+X_2}(t)=G_{X_1}(t)G_{X_2}(t)$ for independence. ## Glossaire - **Random variable**: A function that assigns a real number to each outcome in a sample space. - **Discrete random variable**: A random variable that takes on a countable number of distinct values. - **Probability mass function (PMF)**: A function $p_X(x) = P(X = x)$ for a discrete random variable $X$. - **Cumulative distribution function (CDF)**: $F_X(x) = P(X \leq x)$, the accumulated probability up to $x$. - **Expectation (mean)**: $\mathbb{E}[X] = \sum x \cdot p_X(x)$, the long-run average value of $X$. - **Variance**: $\text{Var}(X) = \mathbb{E}[(X - \mathbb{E}[X])^2]$, measuring the spread of $X$ around its mean. - **Standard deviation**: $\sigma_X = \sqrt{\text{Var}(X)}$, the square root of the variance. - **Mode**: The value of $X$ with the highest probability mass. - **Median**: A value $m$ such that $P(X \leq m) \geq 0.5$ and $P(X \geq m) \geq 0.5$. - **Convolution**: The operation used to find the PMF of the sum of two independent discrete random variables. - **Probability generating function (PGF)**: $G_X(t) = \mathbb{E}[t^X]$, used to compute moments and sums of independent variables. - **Bernoulli distribution**: A discrete distribution with two outcomes, 0 and 1, with probabilities $1-p$ and $p$. - **Binomial distribution**: Models the number of successes in $n$ independent Bernoulli trials. - **Poisson distribution**: Models the number of events occurring in a fixed interval of time or space.