Joint Probability Matrices

Why

We can characterize the dependence of two events in terms of the rank of a particular matrix.

Definition

Given a probability measure $\mathbfsf{P} : \powerset{\Omega } \to \R $ on the finite set $\Omega $ and two events $A, B \subset \Omega $, the joint probability matrix of $A$ and $B$ is the matrix

\[ M = \bmat{ \mathbfsf{P} (A \cap B) & \mathbfsf{P} (A \cap \relcomplement{B}{\Omega }) \\ \mathbfsf{P} (\relcomplement{A}{\Omega } \cap B) & \mathbfsf{P} (\relcomplement{A}{\Omega } \cap \relcomplement{B}{\Omega }) \\ }. \]

Characterization of independence

If $A$ and $B$ are independent, then so are $A$ and $\relcomplement{B}{\Omega }$, $B$ and $\relcomplement{A}{\Omega }$, and $\relcomplement{A}{\Omega }$ and$\relcomplement{B}{\Omega }$. In other words,

\[ M = \bmat{\mathbfsf{P} (A) \\ \mathbfsf{P} (\relcomplement{A}{\Omega }) }\bmat{\mathbfsf{P} (B) & \mathbfsf{P} (\relcomplement{B}{\Omega }}. \]

In this case, we see that $\rank(M) = 1$.

Conversely, suppose $\rank(M) = 1$. Then, using the law of total probabiliy, each row is a multiple of

\[ M1 = \bmat{\mathbfsf{P} (A) \\ \mathbfsf{P} (\relcomplement{A}{\Omega })} . \]

In particular, we have $\mathbfsf{P} (A \cap B) = \alpha \mathbfsf{P} (A)$ and $\mathbfsf{P} (\relcomplement{A}{\Omega } \cap B) = \alpha P(\relcomplement{A}{\Omega })$. So

\[ \mathbfsf{P} (A \cap B) + \mathbfsf{P} (\relcomplement{A}{\Omega } \cap B) = \alpha (\mathbfsf{P} (B) + \mathbfsf{P} (\relcomplement{A}{\Omega })), \]

from which we deduce $\alpha = \mathbfsf{P} (B)$ Likewise, the multiplier for the second column of $M$ is $\mathbfsf{P} (\relcomplement{B}{\Omega })$. In other words, $A$ and $B$ are independent. We conclude that $A$ and $B$ are independent if and only if $\rank(M) = 1$.