Definition

A Markov decision process (MDP) has:

• A finite set of states S.
• A finite set of actions A.
• A transition function T(s,a,s') for all $s\in S$, $a\in A$ ($\Pr(s'\vert s,a)$).
• A reward function R(s,a) for all $s\in S$, $a\in A$.