Bellman Equation for MDPs

For all $s\in S$, $a\in A$,

$$V(s) = \max_{a\in A} \left( R(s,a) + \sum_{s'\in S} T(s,a,s') V(s')\right).$$

This is for ``undiscounted'' MDPs, for reasons that we might not discuss.