Distortion Functions

Why

We want to quantify the error of compressing a real-valued random variable.

Definition

Let $\mathcal{X} $ be a finite set and $q: \R \to \mathcal{X} $ a quantization (see Quantizations) of $\R $. Also, fix a probability space $(\Omega , \mathcal{A} , \mathbfsf{P} )$ and a random variable $x: \Omega \to \R $.

The compression $\hat{x}: \Omega \to \mathcal{X} $ of $x$ under $q$ is $q \circ x$. A distortion function for $x$ and $\hat{x}$ is a function

\[ d: (\Omega \to \R ) \times (\Omega \to \mathcal{X} ) \to \R . \]

Roughly speaking, a distortion function is meant to quantify the error in using this compression.

Examples

The expected mean-squared-error distortion $d_{\text{mse}}$ between $x$ and $\hat{x}$ is

\[ d_{\text{mse}}(x, \hat{x}) = \E [(x - \hat{x})^2] \]

The Kulback-Liebler distortion $d_{\text{kld}}$ defined by

\[ d_{\text{kld}}(x, \hat{x}) = \E [d_{\text{kl}}(\mathbfsf{P} (y \in \cdot \mid x, \hat{x}) \mid \mathbfsf{P} (y \in \cdot \mid \hat{x}))] \]

where $y$ is some random variable that depends on $x$.¹

Future editions will clarify this sentence. ↩︎