Core12 min read

Conditional probability and Bayes' rule

Priors, likelihoods and evidence — how a probability about the world updates when a test comes back positive.

A test is 99% accurate. You test positive. What is the chance you actually have the condition? Almost everyone says 99%. The answer is usually far lower, because the base rate of the condition matters as much as the test. Bayes' rule is the arithmetic that makes this concrete.

Bayes is just conditional probability read backwards

We know $P(+ \mid D)$ — the test's behaviour. We want $P(D \mid +)$ — what the result means. Bayes' rule flips the conditioning by multiplying the prior by the likelihood and dividing by the total probability of the evidence.

Set the base rate, the sensitivity, and the false-positive rate below, then press "another positive test" to fold the posterior back in as the new prior.

A positive test is not a diagnosis — watch the prior do the work

base rate P(D)2.0%sensitivity P(+|D)0.90false positive P(+|¬D)0.080

positive tests: 0

prior

prior P(D): 2.00%
posterior P(D|+): 2.0%

out of 1000 people at this base rate

18 true +

78 false +

precision P(D | +) = 18.0 / (18.0 + 78.4) = 18.7%

Even after a positive, a rare condition can leave the diagnosis more likely false than true.

Bayes says posterior odds = prior odds × likelihood ratio, where the likelihood ratio is sensitivity / false-positive-rate. The small bars show the posterior after each independent test folded back in as the new prior — with a strong test, evidence compounds fast. The frequency strip is the same arithmetic without the notation, and it is why a “99% accurate” test on a rare disease can still be wrong most of the time it fires.

The rule

Conditional probability gives the product rule $P(A \cap B) = P(A \mid B) P(B)$ . Symmetrically, $P(A \cap B) = P(B \mid A) P(A)$ . Equating the two and dividing by $P(B)$ gives Bayes' rule:

$P(A \mid B) = \frac{P(B \mid A)\,P(A)}{P(B)}.$

In the hypothesis-testing vocabulary, $A$ is the hypothesis, $B$ is the observed data, $P(A)$ is the prior, $P(B \mid A)$ is the likelihood, and $P(A \mid B)$ is the posterior. The denominator is computed with the law of total probability over all hypotheses:

$P(B) = P(B \mid A) P(A) + P(B \mid A^c) P(A^c).$

For the disease example this is $P(+) = \text{sens}\cdot P(D) + \text{fpr}\cdot (1 - P(D))$ — the true positives plus the false positives, out of every person tested.

Odds and the likelihood ratio

The same rule is cleaner in odds form. Since $P(A \mid B) / P(A^c \mid B) = \frac{P(B \mid A)}{P(B \mid A^c)} \cdot \frac{P(A)}{P(A^c)}$ :

$\text{posterior odds} = \text{prior odds} \times \text{likelihood ratio}.$

Each piece of evidence multiplies the odds by its likelihood ratio, and independent pieces compound multiplicatively. This is why the widget's "another positive test" button moves the posterior so fast: two independent positive results multiply the odds twice. It also explains why a strong likelihood ratio can overcome a small prior, but only after several observations when the prior is tiny.

The base-rate fallacy

A high sensitivity does not imply a high posterior. With a prevalence of 1% and a test that is 99% sensitive but 5% specific to negatives only, a positive result gives a posterior of about 17%, not 99%. The false positives from the large healthy majority swamp the true positives from the tiny sick minority. Always ask for the base rate before interpreting a test.

Where this leads

The same update underlies Naive Bayes classifiers, where the features are assumed conditionally independent given the class and their likelihood ratios multiply. In a Bayesian model more broadly, the prior distribution is updated to a posterior distribution; the posterior's mean is a natural point estimate, and its spread is a natural measure of how much the data actually told you.

Illustrative vs real

The widget treats sensitivity and the false-positive rate as known constants. In practice they are themselves estimated, often on a population that differs from the one you are testing, and the prior is a judgement call. Real Bayesian work makes those uncertainties explicit; the widget isolates the update rule so you can see the mechanism without them.

Check yourself

Eduspheria wiki · Mathematics for AI, Probability

0 / 5 answered

1Prevalence 1%, sensitivity 99%, false-positive rate 5%. A person tests positive. What is P(D | +)? Give three decimals.
Numeric answer
2In Bayes' rule, P(B | A) is called which of these?
Multiple choice
3What is the name of the factor multiplying the prior odds to produce the posterior odds?
Short answer
4Two independent positive tests always give a larger posterior than one.
True / false
5Prior odds are 1:99 and the likelihood ratio is 10. What are the posterior odds, as a decimal?
Numeric answer

Where next: statistics — describing data, and turning probability models back into estimates.

The rule #

Odds and the likelihood ratio #

Where this leads #

The rule

Odds and the likelihood ratio

Where this leads