Axiomatic probability

The classical definition $P(A) = |A|/|S|$ works only when outcomes are equally likely. For general probability , biased coins, weighted dice, real-world experiments , we need a more flexible framework. That framework is axiomatic probability, formulated by Kolmogorov in 1933.

The axioms

A probability on a sample space $S$ is a function $P$ that assigns to each event (subset of $S$) a real number, satisfying:

Axiom 1 (Non-negativity). $P(A) \ge 0$ for every event $A$.

Axiom 2 (Normalisation). $P(S) = 1$.

Axiom 3 (Additivity). If $A_1, A_2, \dots$ are pairwise mutually exclusive events, then $P(A_1 \cup A_2 \cup \dots) = P(A_1) + P(A_2) + \dots$

For finite sums this is the finite additivity form: $P(A \cup B) = P(A) + P(B)$ when $A \cap B = \varnothing$.

That's it. Three axioms, and every property of probability flows from them.

Immediate consequences

(C1) $P(\varnothing) = 0$.

Proof: $S = S \cup \varnothing$ with $S \cap \varnothing = \varnothing$. By Axiom 3: $P(S) = P(S) + P(\varnothing)$. By Axiom 2: $1 = 1 + P(\varnothing)$. So $P(\varnothing) = 0$.

(C2) Complement. $P(A') = 1 - P(A)$.

Proof: $S = A \cup A'$, $A \cap A' = \varnothing$. By Axiom 3: $P(S) = P(A) + P(A')$. By Axiom 2: $1 = P(A) + P(A')$.

(C3) Monotonicity. $A \subseteq B \Rightarrow P(A) \le P(B)$.

Proof: $B = A \cup (B - A)$ with disjoint pieces, so $P(B) = P(A) + P(B - A) \ge P(A)$.

(C4) $0 \le P(A) \le 1$.

By Axiom 1 and C3 (with $B = S$).

(C5) Inclusion-Exclusion. $P(A \cup B) = P(A) + P(B) - P(A \cap B)$.

This is the general addition rule, valid for any $A, B$ (not just mutually exclusive ones).

Assigning probabilities

To define a probability on a finite sample space $S = \{s_1, s_2, \dots, s_n\}$, just choose non-negative weights $p_1, p_2, \dots, p_n$ with $\sum p_i = 1$. Then $P(s_i) = p_i$, and for any event $A$, $P(A) = \sum_{s_i \in A} p_i$.

For equally likely outcomes, $p_i = 1/n$ , recovering the classical formula.

Worked examples

Example 1. A coin is biased so that $P(H) = 0.6$. Find $P(T)$ and $P(\varnothing)$.

$P(T) = 1 - 0.6 = 0.4$. $P(\varnothing) = 0$.

Example 2. Three events $A, B, C$ have probabilities $P(A) = 0.3, P(B) = 0.4, P(C) = 0.5$ with $A$ and $B$ mutually exclusive, $A$ and $C$ mutually exclusive, but $B \cap C$ has probability $0.2$. Find $P(A \cup B \cup C)$.

$P(A \cup B) = 0.3 + 0.4 = 0.7$. $A \cup B$ is disjoint from $A \cap C = \varnothing$... actually we need to compute $P(A \cup B \cup C)$ using inclusion-exclusion. Using: $P(A \cup B \cup C) = P(A) + P(B) + P(C) - P(A \cap B) - P(A \cap C) - P(B \cap C) + P(A \cap B \cap C)$.

Given $P(A \cap B) = P(A \cap C) = 0$ and $P(B \cap C) = 0.2$ and $P(A \cap B \cap C) = 0$:

$P(A \cup B \cup C) = 0.3 + 0.4 + 0.5 - 0 - 0 - 0.2 + 0 = 1.0$.

Example 3. $P(A) = 0.5, P(B) = 0.3, P(A \cap B) = 0.2$. Find $P(A \cup B)$ and $P(A' \cap B')$.

$P(A \cup B) = 0.5 + 0.3 - 0.2 = 0.6$. $P(A' \cap B') = P((A \cup B)') = 1 - 0.6 = 0.4$.

Example 4. Is $P(A) = 0.7, P(B) = 0.5, P(A \cup B) = 1.1$ a valid probability assignment?

$P(A \cup B)$ must be $\le 1$. But $1.1 > 1$. Not valid.

Example 5. A die is loaded so that the probability of each face is proportional to the face value. Find $P(\text{even})$.

Let $P(i) = c \cdot i$ for $i = 1, \dots, 6$. Sum $= c(1 + 2 + \dots + 6) = 21 c = 1 \Rightarrow c = 1/21$.

$P(\text{even}) = P(2) + P(4) + P(6) = (2 + 4 + 6)/21 = 12/21 = 4/7$.

Try it yourself

1. Valid? $P(A) = 0.4, P(B) = 0.3, A \cap B = \varnothing, P(A \cup B) = 0.7$.
2. Valid? $P(A) = 0.6, P(A') = 0.5$.
3. If $P(A) = 1/3, P(B) = 1/4, P(A \cap B) = 1/6$, find $P(A \cup B)$.
4. If $A \subset B$ and $P(A) = 0.3, P(B) = 0.7$, find $P(B - A)$.
5. $P(A \cup B) = 0.8, P(A) = 0.5$, $P(A \cap B) = 0.2$. Find $P(B)$.
6. A bag has $4$ red, $3$ green, $2$ blue. A ball is drawn. Probability of red? Of not blue?
7. A coin has $P(H) = p$. Find $P(T)$, $P($at least one head in two tosses$)$.
8. Three events have $P(A) = 0.4, P(B) = 0.3, P(C) = 0.2, P(A \cap B) = 0.1, P(B \cap C) = 0.05, P(A \cap C) = 0.06, P(A \cap B \cap C) = 0.02$. Find $P(A \cup B \cup C)$.
9. If $P(A) = 0.5$ and $P(B) = 0.4$, find the maximum and minimum of $P(A \cap B)$.
10. Show $P(A) + P(B) - 1 \le P(A \cap B) \le \min(P(A), P(B))$.
11. A fair die is rolled. Find $P($even or prime$)$.
12. If $P(A) = 0.4, P(A \cup B) = 0.7$, $A$ and $B$ are mutually exclusive, find $P(B)$.

Pitfalls / Tricks

• $P(\varnothing) = 0$, but the converse is not true , events with positive probability can have zero-probability subsets in continuous settings (Class XII).
• Probabilities are always in $[0, 1]$.
• Always check that the assignments you compute satisfy Axiom 2: $P(S) = 1$.
• Insight. The axioms are the minimum required for a consistent theory. From them, everything else follows by algebra.