Decision Processes

Why

We want to talk about making a sequence of decisions.

Definition

Let $S$ and $A$ be finite sets. Let $T: S \times A \to (S \to [0, 1])$ so that for each $s \in S$ and $a \in A$, $T_{sa}: S \to [0, 1]$ is a probability distribution over $S$. We call the ordered triple $(S, A, T)$ a finite state-action process.

A trajectory in the state set $S$ and action set $A$ is a sequence in $S \times A$.

Let $r: S \times A \times S \to \R $, $N \in \N $.

A decision process is a sequence $(S, A, T, r, \gamma , $, consists of two sets, a function set, an action

Other terminology

Decision processes are commonly called markov decision processes.¹

1. As usual, we avoid this terminology in connection with the projects guidelines against using particular names. ↩︎