Module 8 · Lesson 8.4

Bayes' Theorem — Updating Belief With Evidence

Learn how new evidence shrinks the space of possibilities down to a sharper, updated belief.

Intuition

Before you have any evidence, an event's probability is just its share of every possible outcome — the whole circle from earlier lessons. But the moment you learn something new, like a test result, the space of "still possible" outcomes shrinks. You throw away everything inconsistent with the evidence and recompute probability only over what's left. Bayes' theorem is the formula for doing that shrinking correctly.

Core concept

Start with a prior belief — how likely something is before any evidence. Then evidence arrives, like a positive test result. Some of the population that could have produced that evidence actually has the condition, and some doesn't — a positive test can be a true signal or a false alarm. Bayes' theorem asks: out of everyone who could have produced this exact evidence, what fraction of them actually match what you're trying to detect? That updated fraction is the posterior belief.

Have disease, test positive (true+)8
No disease, test positive (false+)9
AnswerP(D|+) = 8/17 ≈ 0.47

Interactive Lab

Drag the base rate and test accuracy sliders. Watch how much of the "positive test" bar is actually disease versus a false alarm.

10%
90%
Before any test10% disease
Among positive tests47% disease

Key Idea

$$ P(D \mid +) = \frac{P(+\mid D)\,P(D)}{P(+\mid D)\,P(D) + P(+\mid \lnot D)\,P(\lnot D)} $$

The numerator is the true positives: people who have the condition and correctly test positive. The denominator adds in the false positives too — everyone else who tests positive without having the condition. Dividing true positives by all positives is exactly the "shrink to what's still possible" idea from the lab above, written as a formula. A spam filter runs the same calculation: given that an email contains the word "free," what fraction of all emails containing that word are actually spam?

$P(D)$The prior — probability of the condition before any evidence
$P(+\mid D)$Sensitivity — chance of a positive test given the condition
$P(+\mid \lnot D)$False positive rate — chance of a positive test without the condition
$P(D \mid +)$The posterior — updated probability after seeing a positive test

Think Further

  1. Even with a 90%-accurate test, why can a positive result still mean less than a 50% chance of actually having a rare condition?
  2. If the same test were used for a much more common condition, what would happen to the posterior probability, and why?
  3. How does this same "shrinking space of possibilities" idea apply to a spam filter that sees the word "free" in an email?
Show suggested answers
  1. When a condition is rare, most of the population is healthy, so even a small false-positive rate applied to that huge healthy group can generate more false alarms than true detections among the much smaller group who actually have the condition.
  2. The posterior probability would rise. With more people truly having the condition to begin with, a larger share of all positive results come from genuine cases rather than false alarms.
  3. The filter treats "contains the word free" as evidence, then asks what fraction of all emails containing that word are actually spam — narrowing from "all emails" down to "emails with this word," then further down to "spam among those," is exactly the same shrinking-space calculation.