Minimum Mean Squared Error Estimator

Why

What is the best estimator for a real-value random variable if we consider the squared loss.

Definition

We want to estimate a random variable $x: \Omega \to \R ^n$ from a random variable $y: \Omega \to \R ^n$ using an estimator $\phi : \R ^m \to \R ^n$.

Let $x:\Omega \to\R ^n$ and $y: \Omega \to \R ^m$. A minimum mean squared error estimator or MMSE estimator or least square estimator for $x$ given $y$ is an estimator $f: \R ^m \to \R ^n$ which minimizes $\E \norm{f(x) - y}^2$.