Classifiers

Why

We often want to predict one of several outcomes.

Definition

A classifer is a predictor whose codomain is a finite set. In this case, we call the codomain the label set and we call its elements classes (or labels, categories). We call the prediction of a classifier on an input a classification.

If the set of labels has two elements, then we call the classifier a binary classifier (or two-way classifier, two-class classifier, boolean classifier). In the case that there are $k$ labels, we call the classifier a $k$-way classifier (or $k$-class classifier, multi-class classifier). The second term is meant to indicate, not that the classifier assigns to each point several classes, but that the classification decision is made between several classes.

Basic examples

Let $A$ be a set of inputs and let $B$ be a set of labels. Define $B = \set{0, 1}$ (or $\set{-1,1}$, {False, True}, {Negative, Positive}). Then $B$ is finite with two elements and $f: A \to B$ is a binary classifier with labels $0$ and $1$.

If the case $B = $ $\{$No, Maybe, Yes$\}$, we call $f: A \to B$ a three-way classifier. Other examples for $B$ include a list of languages, the set of English words in some dictionary, or the set of $m!$ possible orders of $m$ horses in a race.

When dealing with a finite set of $k$ arbitrary objects, it is often convenient to take associate the objects with the first $k$ positive integers and take the set $B = \set{1, \dots , k}$ as the set of labels; here $k \in \N $.

Other terminology

Following our terminology, but speaking of processes, some authors refer to the application of inductors for these special cases as binary classification and multi-class classification. Or they speak of classification and classification problems. Roughly speaking, a classifier classifies all inputs into categories.

Alternatively, some authors (especially in the statistics literature) refer to a classifier as a discriminator and reference discrimination problems.