TOPIC 2 / 3

Expectation, Variance & the CLT

Useful preparation: Random Variables, PMFs, PDFs & CDFs

For a long equation, swipe sideways inside its formula box to see every term.

Meaning & notation

Expectation is a probability-weighted balance point, not a promise about any single observation. Variance measures squared distance from that balance point. Averaging independent observations preserves their common expectation but reduces their variability. The central limit theorem adds a statement about the shape of the distribution of the standardized average, under specific assumptions.

Expectation: The mean mu=E[X] is the sum of x times its probability mass, or the integral of x times its density, when the expectation exists. E[g(X)] weights g(x) using the same distribution.
Variance: Variance is E[(X-mu) squared], equivalently E[X squared]-mu squared when the second moment is finite. Standard deviation is its square root and has the original measurement units.
Standard error: The standard error of an estimator is its sampling standard deviation. For the mean of n independent identically distributed observations with variance sigma squared, it is sigma divided by the square root of n.

Conditions before calculation

Linearity of expectation does not require independence, but the relevant expectations must exist. Nonlinear functions generally cannot be moved outside expectation.
Variances of independent variables add. For dependent variables, covariance terms must be included; multiplying a random variable by a constant multiplies its variance by that constant squared.
The classical central limit theorem assumes independent identically distributed observations with finite positive variance. No fixed sample-size threshold guarantees a good normal approximation for every distribution.

Why the method works

1. Calculate moments with weights

First compute E[X], then E[X squared]. Squaring the mean after averaging is a different operation from averaging squared observations.

\operatorname{Var}(X)=E[X^2]-(E[X])^2

2. Track scaling

Adding a constant shifts every outcome equally, leaving spread unchanged. Multiplying distances by a multiplies squared distances by a squared.

E[aX+b]=aE[X]+b,\qquad \operatorname{Var}(aX+b)=a^2\operatorname{Var}(X)

3. Distinguish exact facts from approximation

The mean and variance of the sample mean are exact under the stated assumptions. Its approximate normal shape is the additional asymptotic conclusion.

E[\bar X]=\mu,\quad \operatorname{Var}(\bar X)=\frac{\sigma^2}{n},\quad \frac{\sqrt n(\bar X-\mu)}{\sigma}\xrightarrow{d}N(0,1)

Worked examples

EXAMPLE 1

X takes values 0, 2, and 5 with probabilities 0.2, 0.5, and 0.3. Find its mean and variance.

The probabilities sum to one. Compute the weighted mean.
$E[X]=0(0.2)+2(0.5)+5(0.3)=2.5$
Square outcomes before weighting to find the second moment.
$E[X^2]=0+4(0.5)+25(0.3)=9.5$
Subtract the square of the mean.
$\operatorname{Var}(X)=9.5-2.5^2=3.25$

Conclusion: The mean 2.5 need not be a possible observed value.

EXAMPLE 2

Independent repeated measurements have mean 40 and standard deviation 6. Find the spread of an average of 36 measurements, and describe a normal approximation.

The average retains expectation 40 by linearity.
Independence adds the 36 variances before division by 36 squared.
$\operatorname{Var}(\bar X)=\frac{36\cdot6^2}{36^2}=1,\qquad \operatorname{SE}(\bar X)=1$
If the normal approximation is adequate, standardizing 38 and 42 gives -2 and 2.
$P(38\le\bar X\le42)\approx P(-2\le Z\le2)\approx0.9545$

Conclusion: The standard error is exactly 1; the probability approximation requires information about the underlying distribution and is exact in form if the measurements are normal.

Pitfalls & counterexamples

Tempting claim: The variance of 3X is three times the variance of X.

Correction: Variance scales by the square of the multiplier.

Test the claim: If X is equally likely to be -1 or 1, its variance is 1, whereas 3X has variance 9.

Check your understanding

Does increasing sample size make the original observations normally distributed?

Write a reason before opening the explanation. This reflection is not graded or saved.

Compare your reasoning

No.

The central limit theorem concerns the standardized mean, not a change in the population distribution.

Connect this idea

Standard errors quantify sampling uncertainty in confidence intervals and tests. Dependence, heavy tails, and biased sampling can prevent familiar averaging formulas from answering the intended question.

Practise this topic →Review concept flashcards

Reference reading

Use these texts to study the underlying definitions and theorems in greater depth. The explanations and examples above are written for Numerive.