Joint Probability Distributions

Why

Suppose we flip a coin $n$ times. How should we define a probability distribution on $\set{0, 1}^n$. Suppose instead we flip $n$ different coins, each once. How now?

Discussion

If we model the outcome that coin $i$ lands heads as $1$ and that it lands tails by $0$, then we are asking to treat of uncertain outcomes from the set $\set{0, 1}^n$ of length-$n$ sequences.

Here $x \in \set{0, 1}^n$ could correspond to the outcome of flipping one coin $n$ times or the outcome of flipping $n$ different coins. This is a simple example of a frequent and obvious phenomenon in mathematics wherein the same mathematical model (here the outcome set) can be used to model different situations in the real world (here flipping the same coin several times, or flipping several different coins at once). Compare with numbers, etc.

Definition

Consider a distribution over a product of $n$ sets, where the product is indexed by the first $n$ natural numbers. We call the distribution a joint distribution with $n$ components.