Differential Cross Entropy
Definition
The
differential cross entropy
of a second density with respect to a
a first density
is the integral of the second density
against the negative log
of the first density.
Let $R$ denote the set of
real numbers.
Let $f: R^n \to R$ and
$g: R^n \to R$ be
probability density functions.
The differential cross entropy of $f$
relative to g
\[
- \int g \log f
\]