Numerical Characteristics of Random Variables
This article introduces numerical characteristics of random variables such as expectation and variance.
Expectation
Definition
Discrete Random Variable
Let the probability distribution of a discrete random variable \(X\) be \(p_i = P\{ X = x_i \}\). If the sum
converges absolutely, then its value is called the expectation of \(X\), denoted as \(EX\).
Continuous Random Variable
Let the density function of a continuous random variable \(X\) be \(f(x)\). If the integral
converges absolutely, then its value is called the expectation of \(X\), denoted as \(EX\).
Unified Definition
Let the distribution function of random variable \(X\) be \(F(x)\). If the Stieltjes integral
converges absolutely, then its value is called the expectation of \(X\), denoted as \(EX\).
Example of Non-existent Expectation
Consider a discrete random variable \(X\) with the following distribution:
Although the sum \(\sum x_i p_i\) converges to \(- \ln 2\), it is not absolutely convergent, so the expectation of \(X\) does not exist.
Next, consider a continuous random variable \(Y\) with the following density function:
It can be easily verified that the expectation of \(Y\) also does not exist.
Properties of Expectation
Linearity
If the expectations of random variables \(X, Y\) exist, then
- For any real numbers \(a, b\), \(E(aX + b) = a \cdot EX + b\).
- \(E(X + Y) = EX + EY\).
Expectation of Product of Random Variables
If the expectations of random variables \(X\),\(Y\) exist and \(X\),\(Y\) are independent, then
Note: Independence in the above property is not a necessary condition.
Counterexample
Consider random variables \(X\) and \(Y\), where \(X\) follows a uniform distribution on \([-1, 1]\), and \(Y = X^2\).
Conversion Between Expectation and Probability
For a random event \(A\), consider its indicator function \(I_A\):
According to the definition, its expectation can be found as \(EI_A = P(A)\). This conversion is very common in practical applications.
Example
Suppose for a sequence \(\{ a_i \}\) of length \(n\), \(a_k\) takes the value \(k\) with probability \(p_k\), and takes \(0\) with probability \(1 - p_k\). Consider how to find the expectation of \(S = \sum_{i=1}^{n} a_i\).
If using the definition directly, one needs to find the probability of \(S\) at each possible value, which is a cumbersome calculation process and will not be elaborated here.
On the other hand, let \(I_k\) denote the indicator function of the random event \(a_k = k\). Then
Furthermore, it is not difficult to find
Conditional Distribution and Conditional Expectation
We have previously studied conditional probability; similarly, the concept of conditional expectation can be introduced.
Definition
For two random variables \(X\),\(Y\), the probability distribution (density function) of \(X\) given \(Y = y\) is called the conditional probability distribution (conditional probability density), denoted respectively as
Under this condition, the expectation of \(X\) is called the conditional expectation, denoted as \(E[X|Y=y]\).
Properties of Conditional Expectation
Many properties of conditional expectation can be inferred from conditional probability and will not be detailed here.
It is worth mentioning that \(E[X | Y]\) is generally a function of the random variable \(Y\), and this function is usually not linear. However, in fact,
The above formula is called the law of total expectation.
Application
HDU 5984 Pocky
There is a Pocky of length \(L\). Each time it is randomly broken into two segments. If the length of the right segment is not greater than \(d\), the process stops; otherwise, repeat the above process on the right segment. Find the expected number of repetitions.
Solution
Let \(f(x)\) denote the expected number of repetitions for a segment of length \(x\). The case \(x \leq d\) is trivial.
When \(x > d\), suppose the breaking point is at a distance \(k\) from the right end. Clearly, \(k \sim U[0, x]\), and the expected number of repetitions at this time is
By the law of total expectation,
Solving this integral equation and substituting the initial condition yields
Variance
Definition
Let the expectation \(EX\) of random variable \(X\) exist, and the expectation
also exist. Then the value of the above expression is called the variance of random variable \(X\), denoted as \(DX\) or \(Var(x)\). The arithmetic square root of the variance is called the standard deviation, denoted as \(\sigma(X) = \sqrt{DX}\).
Properties of Variance
If the variance of random variable \(X\) exists, then
- For any constants \(a, b\), \(D(aX + b) = a^2 \cdot DX\).
- \(DX = E(X^2) - (EX)^2\).
Covariance and Correlation Coefficient
Generally, the equality \(D(X + Y) = DX + DY\) does not hold. We naturally ask two questions:
- What exactly is the difference between \(D(X + Y)\) and \(DX + DY\)?
- Under what conditions does \(D(X + Y) = DX + DY\) hold?
For the first question, we introduce covariance as an answer.
Definition of Covariance
For random variables \(X, Y\), the expression
is called the covariance of \(X\) and \(Y\), denoted as \(\operatorname{Cov}(X, Y)\).
Properties of Covariance
For random variables \(X, Y, Z\), we have
- \(\operatorname{Cov}(X, Y) = \operatorname{Cov}(Y, X)\).
- For any constants \(a, b\), \(\operatorname{Cov}(aX + bY, Z) = a \cdot \operatorname{Cov}(X, Z) + b \cdot \operatorname{Cov}(Y, Z)\).
Covariance and variance are also related as follows:
- \(DX = \operatorname{Cov}(X, X)\).
- \(D(X + Y) = DX + 2 \operatorname{Cov}(X, Y) + DY\).
About Covariance
You may notice that the properties of covariance are highly consistent in form with the operational properties of the vector inner product.
From the perspective of functional analysis, for a given probability space, all random variables on it form a linear space, and covariance is an inner product on this space, while the standard deviation is the norm induced by this inner product.
For the second question raised earlier, it is easy to see that \(D(X + Y) = DX + DY\) if and only if \(\operatorname{Cov}(X, Y) = 0\). An intuitive necessary condition is that \(X\) and \(Y\) are independent, because then
But this condition is not sufficient. To describe the relationship between random variables \(X\),\(Y\) satisfying \(\operatorname{Cov}(X, Y) = 0\), we introduce the correlation coefficient.
Correlation Coefficient
For random variables \(X, Y\), the expression
is called the Pearson correlation coefficient of \(X\) and \(Y\), denoted as \(\rho_{X,Y}\).
The Pearson correlation coefficient describes the strength of the linear association between two random variables. The larger \(|\rho_{X,Y}|\) is, the stronger the linear association between \(X\) and \(Y\). It can be proven that \(|\rho_{X,Y}| \leq 1\), and \(|\rho_{X,Y}| = 1\) can only occur in the following two cases:
- When there exist a real number \(a\) and a positive real number \(b\) such that \(P(X = a + bY) = 1\), then \(\rho_{X,Y} = 1\);
- When there exist a real number \(a\) and a negative real number \(b\) such that \(P(X = a + bY) = 1\), then \(\rho_{X,Y} = -1\).
When \(\rho_{X,Y} = 0\), we say random variables \(X\) and \(Y\) are uncorrelated. In this case, there is no linear relationship between \(X\) and \(Y\).
「Uncorrelated」and「Independent」
Two random variables being uncorrelated only indicates that there is no linear association between them, and does not imply the absence of other forms of relationship.
Therefore, two random variables \(X, Y\) being uncorrelated is a necessary but not sufficient condition for them to be independent.
For the second question raised at the beginning of this section, we give the conclusion: the necessary and sufficient condition for \(\operatorname{Cov}(X, Y) = 0\) is that one of \(X\),\(Y\) takes a constant value with probability \(1\), or \(X, Y\) are uncorrelated.
本页面最近更新：，更新历史
发现错误？想一起完善？ 在 GitHub 上编辑此页！
本页面贡献者：OI-wiki
本页面的全部内容在 CC BY-SA 4.0 和 SATA 协议之条款下提供，附加条款亦可能应用