Covariance
Definition
The covariance between two random variables which are each integrable and whose product is integrable is the expectation of their product less the product of their expectation.
Notation
Let $f$ and $g$ be two integrable random
variables with $fg$ integrable.
Denote the covariance of $f$ with $g$ by
$\cov(f, g)$.
We defined it:
\[
\cov(f, g) = \E (fg) - \E (f)\E (g).
\]
Properties
Covariance is symmetric and billinear.
The covariance of a random variable with itself
is its variance.
Let $f$ be a square-integrable real-valued
random variable, then
\[ \cov(f, f) = \E (ff) - \E (f)\E (f) = \E (f^2) - (\E (f))^2 = \var(f). \]
The variance of a sum of integrable real-valued
random variables whose pairwise products are
integrable is the double sum of the pairwise
covariances.
Let $f_1, \dots , f_n$ be integrable random
variables with $f_if_j$ integrable for all $i,j
= 1, \dots , n$.
Using the billinearity,
\[ \begin{aligned} \textstyle \var \sum_{i = 1}^{n} f_i &= \cov(\sum_{i = 1}^{n} f_i, \sum_{i = 1}^{n}f_i) \\ &= \sum_{i = 1}^{n} \sum_{j = 1}^{n} \cov(f_i, f_j) \end{aligned} \]