Supervised Learning Algorithms

Why

We sometimes use special language for a function inductor, which alludes to its similarities with “learning.”

Definition

Let $\CU$ and $\CV$ be sets and let $\set{G_n: (\CU \times \CV)^n \to \powerset{X \times Y}}_{n \in \N }$ be a family of functional inductors.

Suppose we have a dataset $(u^1, v^1), \dots , (u^n, u^n) \in \CU \times \CV$, and we use our inductor (i.e., learning algorithm) to produce a predictor $f: \CU \to \CY$ defined by $f \equiv G_m(a^1, \dots , a^m)$. Thus, we have a sequence of predictors $f^1, \dots , f^n$. We can evaluate our learning algorithm on th A predictor can be used to “guess” inputs which do not necessarily appear in the dataset. For this reason, some authors call an inductor (or family of inductors) a learner or learning algorithm. In accordance with this usage, they refer to the argument of an inductor as the training data or training dataset or training set. As with our terminology dataset, the word “set,” however, may mislead since since we are speaking of a sequence.

It is common to refer to the construction a predictor from a dataset a learning problem. In this case, the learning problem is said to be supervised learning. By supervision, we mean to indicate that we have the outputs corresponding to the inputs. In line with this usage, the outputs are often called labels and the labels are said “supervise” or “to provide supervision.” Since a predictor can be used to guess the output of an input, some authors call an inductor (or family of inductors) a learner or learning algorithm or supervised learning algorithm and refer to the argument as the training dataset. Often the word “supervised” is included, as in supervised learning. This language intends to indicate that inputs are given along with outputs, and these outputs “provide supervision to the algorithm.”