\(\DeclarePairedDelimiterX{\Set}[2]{\{}{\}}{#1 \nonscript\;\delimsize\vert\nonscript\; #2}\) \( \DeclarePairedDelimiter{\set}{\{}{\}}\) \( \DeclarePairedDelimiter{\parens}{\left(}{\right)}\) \(\DeclarePairedDelimiterX{\innerproduct}[1]{\langle}{\rangle}{#1}\) \(\newcommand{\ip}[1]{\innerproduct{#1}}\) \(\newcommand{\bmat}[1]{\left[\hspace{2.0pt}\begin{matrix}#1\end{matrix}\hspace{2.0pt}\right]}\) \(\newcommand{\barray}[1]{\left[\hspace{2.0pt}\begin{matrix}#1\end{matrix}\hspace{2.0pt}\right]}\) \(\newcommand{\mat}[1]{\begin{matrix}#1\end{matrix}}\) \(\newcommand{\pmat}[1]{\begin{pmatrix}#1\end{pmatrix}}\) \(\newcommand{\mathword}[1]{\mathop{\textup{#1}}}\)
Needs:
Discrete Entropy ›
Real-Valued Random Variable Expectation ›
Prefix-Free Codes ›
Needed by:
None.
Links:
Sheet PDF ›
Graph PDF ›

Optimal Average Codeword Length

Why

Where does entropy come from?1

Result

\[ H(x) \leq \min_{c: \text{prefix free}} \E [\text{len}(c(x)] \leq H(x) + 1 \]

  1. Futre editions will expand ↩︎
 Copyright © 2023 The Bourbaki Authors — All rights reserved — Version 13a6779cc — About — Show the old page view