Wiki
Core11 min read

Covariance and correlation: how two variables move together

Covariance measures joint variation, correlation rescales it into [−1, 1], and neither one implies causation.

A single variable has a location and a spread. Two variables have a relationship: when one rises, does the other tend to rise with it, move against it, or ignore it? Covariance and correlation answer that question, and the gap between correlation and causation is the most consequential caveat in all of statistics.

Correlation is the cosine of centered data

Center both variables by subtracting their means and think of the resulting lists as vectors. Their cosine is exactly Pearson's rr. That is why rr is unitless and always lies between −1 and 1: it contains the angle, not the lengths.

Drag the target correlation below. The axis that matters is the measured Pearson rr, which is recomputed from the plotted points, not copied from the slider.

One dial reshapes the cloud — and r is measured, not assumed

Pearson r
0.769
r²
0.591
cov(X,Y)
1.650
slope
0.777

Drag ρ toward ±0.95 and the points collapse onto a line; near zero they become a blob.

Pearson r is covariance rescaled by both standard deviations, so it is unitless and always in [−1, 1]. It is the cosine of the angle between the centered data vectors, and it is exactly what a correlation coefficient claims and no more: the parabola has a perfect deterministic relationship and r = 0, because its symmetry cancels.

Covariance

The population covariance of two random variables is

Cov⁡(X,Y)=E[(X−E[X])(Y−E[Y])]=E[XY]−E[X]E[Y].\operatorname{Cov}(X, Y) = E\big[(X - E[X])(Y - E[Y])\big] = E[XY] - E[X]E[Y].

The sample version replaces expectations with averages. The sign tells the direction of the relationship: positive when large XX goes with large YY, negative when it goes with small YY, near zero when there is no linear co-movement. The magnitude, however, depends on the units of XX and YY, so a covariance of 1000 might be weak or strong depending on the scales involved. That unit-dependence is exactly what correlation fixes.

Useful identities:

  • Cov⁡(X,X)=Var⁡(X)\operatorname{Cov}(X, X) = \operatorname{Var}(X).
  • Cov⁡(aX+b, cY+d)=ac Cov⁡(X,Y)\operatorname{Cov}(aX + b,\ cY + d) = ac\,\operatorname{Cov}(X, Y).
  • Var⁡(X+Y)=Var⁡(X)+Var⁡(Y)+2Cov⁡(X,Y)\operatorname{Var}(X + Y) = \operatorname{Var}(X) + \operatorname{Var}(Y) + 2\operatorname{Cov}(X, Y).
  • Independent variables have zero covariance (the converse is false).

Pearson correlation

Dividing covariance by both standard deviations removes the units:

r=Cov⁡(X,Y)σXσY,−1≤r≤1.r = \frac{\operatorname{Cov}(X, Y)}{\sigma_X \sigma_Y}, \qquad -1 \le r \le 1.

r=+1r = +1 means the points lie exactly on an upward line; r=−1r = -1 on a downward line; r=0r = 0 means no linear co-movement. The square r2r^2 is the fraction of the variance in one variable explained by a linear fit to the other, which is the bridge to regression and to R2R^2.

Correlation is not causation — and not even full dependence

Two failures to keep straight:

  • Causation. Two variables can be strongly correlated because a third variable drives both, or by coincidence. Correlation constrains the joint distribution; it does not license an intervention-based reading.
  • Linearity. Correlation measures linear association only. Toggle the parabola in the widget: the relationship is deterministic and perfect, yet rr is near zero because the symmetry cancels. Datasets with strong nonlinear structure can look uncorrelated, which is why you always plot.

Zero correlation does not mean independence

Independence implies zero correlation, but zero correlation does not imply independence. The two coincide only for jointly normal variables (and a few other special cases). A U-shaped relationship is the classic counterexample: perfectly dependent, zero correlation.

Illustrative vs real

Pearson rr is sensitive to outliers and only captures linear structure. In practice you would also compute a rank correlation (Spearman or Kendall) and inspect the scatter before concluding anything. The widget generates clean Gaussian-ish points; real data has clusters, missing values and outliers that can flip the sign of rr on their own.

Check yourself

Eduspheria wiki · Mathematics for AI, Statistics

0 / 5 answered

  1. 1σ_X = 2, σ_Y = 5 and r = 0.8. What is Cov(X, Y)?
    Numeric answer
  2. 2A scatter plot shows a perfect U-shaped relationship. What is the likely value of Pearson's r?
    Multiple choice
  3. 3What quantity does dividing Cov(X, Y) by σ_X·σ_Y produce?
    Short answer
  4. 4If Cov(X, Y) = 0, then X and Y are independent.
    True / false
  5. 5Var(X) = 4, Var(Y) = 9 and Cov(X, Y) = 3. What is Var(X + Y)?
    Numeric answer

Where next: estimation — using data to estimate unknown quantities, and the orthogonality principle that makes least squares optimal.