Posterior Event Probabilities

Result

Suppose $P$ is a probability measure on a finite set of outcomes $\Omega $. For any two events $A, B$ with $P(A), P(B) > 0$, we have

\[ P(A \mid B) = \frac{ P(B \mid A)P(A) }{ P(B) }. \]

By definition, we have

\[ P(A \mid B) = \frac{ P(A \cap B) }{ P(B) }. \]

And also symmetrically,

\[ P(B \mid A) = \frac{ P(A \cap B) }{ P(A) }. \]

From this second equation we have $P(A \cap B) = P(B \mid A) P(A)$, which we can substitute into the numerator of the first expression to obtain the result.

This result is known by many names including Bayes’ rule, Bayes rule (no possessive), Bayes’ formula, and Bayes’ theorem.

It is a basic consequence of the definition of conditional probability, but it is useful in the case that we are given problem data in terms of the probabilities on the right hand side of the above equation.

Examples

Diagnostic test

Suppose we want to model the situation in which a rare disease afflicts 0.5% of a population and we have a diagnostic test that is 99% accurate.

We consider the population $\Omega $ of people. We agree to partition the population into $D$ and $H$ so that

\[ D \cup H = \Omega \quad \text{ and } \quad D \cap H = \varnothing \]

$D$ is the set of people with the disease, and $H$ is the set of healthy people without the disease. Similarly, we agree to partition the population into $R$ and $N$ so that

\[ R \cup N = \Omega \quad \text{ and } \quad P \cap N = \varnothing \]

$R$ is the set of people who test positive, and $N$ is the set of people who test negative.

We agree that 0.5% of the the population being afflicted means, $P(D) = 0.005$. We agree that having a 99% accurate test means means

\[ P(D \mid R) = 0.99 \quad \text{ and } P(H \mid N) = 0.99 \]

Now, what is the conditional probability of having the disease given that the test is positive? Using Bayes rule,

\[ P(D \mid R) = \frac{P(R \mid D)P(D)}{P(R\mid D)P(D) + P(R\mid H)P(H)} \]

Using our supposed values,

\[ P(D \mid R) = \frac{0.99 \cdot 0.005}{0.99 \cdot 0.005 + 0.01 \cdot 0.995} \approx 0.33 \]

This may be viewed as suprising, since the test is perceived to be accurate.

The frequentist interpretation is clear: if have many outcomes, say a thousand individuals, we may expect that about five of these thousand have the disease. The test is likely to diagnose these correctly. However, of the other 995 people, about 1% of them—say 10 people—will be misdiagnosed. Thus we may expect to see about 15 positive test results, but only five of which correspond to individuals with the disease.

Compound form

More is true.

Suppose $P$ is a finite probability measure on a set of outcomes $\Omega $. For any three events $A, B, C$ with $P(A), P(B), P(C) > 0$, we have

\[ P(A \mid B \cap C) = \frac{ P(B \mid A \cap C)P(A \mid C) }{ P(B \mid C) }. \]

Future editions will include, the strategy is the same as above.