Core12 min read

Expectation and variance: the balance point and the spread

Expectation is the long-run average and a linear operator; variance measures spread — and Chebyshev turns that spread into a bound.

The full distribution of a random variable is often more than you need. Two numbers — where the mass sits on average, and how far it spreads — carry most of the useful information. Draw samples, average them, and you approach the first; look at the typical squared distance from it and you measure the second.

Expectation is a weighted average and a seesaw

For a discrete variable, $E[X] = \sum_k k\, p(k)$ : each value weighted by how likely it is. It is the balance point of the distribution. Change the weights and the balance point slides — that is the whole of the widget below.

Reweight the six payoffs and watch the exact mean and variance recompute while a stream of actual samples settles onto the mean.

Reweight the outcomes — expectation is the balance point

X=10.17X=20.17X=30.17X=50.17X=80.17X=130.17

sample mean: 5.333

E[X]: 5.333
Var(X): 16.889
σ: 4.110

The dashed line is E[X] = Σ x·P(X = x), and the jagged line is the mean of the samples so far. Each run is a different random walk, but they all drift toward the same balance point — the law of large numbers. Variance is the average squared distance from that point; pushing the weights to the extremes is the cheapest way to feel what “spread” means.

Expectation

For a discrete variable, $E[X] = \sum_k k\, p(k)$ ; for a continuous one, $E[X] = \int x f(x)\,dx$ . The single most useful property is linearity, which holds with no independence assumption whatsoever:

$E[aX + bY + c] = a\,E[X] + b\,E[Y] + c.$

For a function of $X$ , $E[g(X)] = \sum_k g(k) p(k)$ — you do not need the distribution of $g(X)$ , only the distribution of $X$ . This is the law of the unconscious statistician, and it is what makes computing moments mechanical.

Note carefully that in general $E[XY] \neq E[X] E[Y]$ . Equality holds when $X$ and $Y$ are independent, and the gap between the two sides is the covariance.

Variance and standard deviation

Variance measures the average squared distance from the mean:

$\operatorname{Var}(X) = E\big[(X - E[X])^2\big] = E[X^2] - (E[X])^2.$

The second form is the one you compute with: square the mean and subtract it from the mean of the square. Standard deviation is $\sigma = \sqrt{ \operatorname{Var}(X)}$ , which restores the original units. Variance is quadratic in scale:

$\operatorname{Var}(aX + b) = a^2 \operatorname{Var}(X),$

so shifting a distribution does not change its spread and scaling by $a$ multiplies the variance by $a^2$ . For sums of independent variables variances add: $\operatorname{Var}(X + Y) = \operatorname{Var}(X) + \operatorname{Var}(Y)$ . Without independence the covariance term appears.

The inequalities

Two bounds let you reason about tails knowing only the mean and variance:

Markov. For a non-negative $X$ and $a > 0$ , $P(X \ge a) \le E[X] / a$ .
Chebyshev. For any $X$ with finite variance and $k > 0$ ,

$P\big(|X - E[X]| \ge k\sigma\big) \le \frac{1}{k^2}.$

Chebyshev is loose — a normal variable is far more concentrated than $1/k^2$ — but it requires only that the variance exists. That is enough to prove the law of large numbers: the sample mean's variance shrinks like $1/n$ , so it concentrates on the true mean as $n$ grows. The jagged line in the widget is exactly this convergence in action.

Expectation need not exist

The expectation is an integral, and integrals can diverge. A Cauchy distribution has no mean at all, and heavy-tailed losses can have infinite variance. "The expected value is infinite" is a real statement, not a rounding error; when it happens, simulation averages wander without settling. Always check the tail before trusting a mean.

Illustrative vs real

The widget samples a six-point payoff with weights you can dial, which makes the law of large numbers visible in a second. Real estimators may have infinite variance, correlated samples, or heavy tails, and then the sample mean converges far more slowly — or not at all. The convergence seen here is a property of this finite-variance toy, not a universal guarantee.

Check yourself

Eduspheria wiki · Mathematics for AI, Probability

0 / 5 answered

1A fair six-sided die is rolled once. What is E[X]?
Numeric answer
2Var(X) = 4. What is Var(3X + 7)?
Numeric answer
3For which pair of variables is E[XY] = E[X]·E[Y] guaranteed?
Multiple choice
4Which theorem says the sample mean converges to the population mean as the sample size grows?
Short answer
5Chebyshev's inequality requires the distribution to be normal.
True / false

From the mid-term paper

Modeled on NITJ AI-501, Mid-Term October 2024

0 / 5 answered

1Ten daily temperature readings (in degrees Celsius) are 33.5, 39.2, 36.7, 33.5, 32.9, 37.4, 34.1, 35.4, 30.9 and 37.2. What is the mean?
Numeric answer
2For the same ten readings, what is the median?
Numeric answer
3For the same ten readings, what is the mode?
Numeric answer
4The matrix [[2, 8], [7, 9]] has two eigenvalues. Their sum equals the trace of the matrix. What is that sum?
Numeric answer
5Which method projects data onto a direction that maximizes the ratio of between-class scatter to within-class scatter?
Short answer

Where next: Bayes' rule — updating a probability when new evidence arrives.

Expectation #

Variance and standard deviation #

The inequalities #

Expectation

Variance and standard deviation

The inequalities