Bayes' Theorem
Bayes' Theorem is a fundamental formula in probability theory and statistics that describes how to update the probability of a hypothesis based on new evidence.
Definition
Bayes' Theorem is mathematically expressed as:
$$ P\left(A \vert B\right) = \frac{P\left(B \vert A\right)P\left( A \right)}{P\left( B \right)} $$
Where:
- \(P(A \vert B)\): Posterior probability. The probability of the hypothesis \(A\) given the event \(B\).
- \(P(B \vert A)\): Likelihood. The probability of observing \(B\) if \(A\) is true.
- \(P(A)\): Prior probability. The initial probability of the hypothesis \(A\), before observing \(B\).
- \(P(B)\): Marginal probability. The total probability of event \(B\), considering all possible hypotheses.
Intuition
Bayes' Theorem allows us to update our initial belief \((P(H))\) in a hypothesis based on how well new data \((E)\) aligns with that hypothesis. It provides a logical framework for evidence-based learning.
Practical Example
Imagine a medical test detects a disease with \(95\ \%\) accuracy \((P(positive \vert disease) = 0.95)\). However, the disease affects only \(1\ \%\) of the population \((P(disease) = 0.01)\). Suppose a person receives a positive test result. What is the probability they actually have the disease \((P(disease \vert positive))\)?
Using Bayes' Theorem:
\(P(disease \vert positive) = \frac{P(positive \vert disease) \cdot P(disease)}{P(positive)}\)
Where:
\(P(positive) = P(positive \vert disease) \cdot P(disease) + P(positive \vert no\ disease) \cdot P(no\ disease)\)
You can calculate this to find that the probability is often much lower than one might intuitively expect.
Bayes' Theorem Calculator
Instructions:
Use the boxes to enter the necessary data as indicated following the Bayes Theorem equation:
Posterior probability \(P\left(A \vert B\right)\):
Bayes' Theorem Simulator
Instructions:
Move the sliders \(P\left( A \right),\ P\left( B \vert A^{\complement} \right),\ P\left( B^{\complement} \vert A \right) \), prior probability, false positive rate and false negative rate respectively, to change the probability values in the chart.
Evidence probability
To calculate the evidence probability
$$P \left( B \right) = P\left( B \cap A \right) + P\left( B \cap A^{\complement} \right)$$
$$P\left( B \cap A \right) = P\left( B \vert A \right) P\left(A\right)$$
$$P\left( B \cap A^{\complement} \right) = P\left( B \vert A^{\complement} \right) P \left( A^{\complement} \right)$$
$$\Rightarrow P \left( B \right) = P\left( B \vert A \right) P \left( A \right) + P\left( B \vert A^{\complement} \right) P \left( A^{\complement} \right)$$
As the complement of a probability \(P\) is defined as \(1-P\)
$$ P\left( B \vert A \right) = 1 - P\left( B^{\complement} \vert A \right) $$
$$P \left( A^{\complement} \right) = 1 - P\left( A \right)$$
$$\Rightarrow P \left( B \right) = \left( 1 - P\left( B^{\complement} \vert A \right) \right) P \left( A \right) + P\left( B \vert A^{\complement} \right) \left(1 - P\left( A \right)\right)$$