Homogeneous Markov Decision Processes

Homogeneous Markov Decision Processes


  • Matthew Sobel
  • Wei Wei


Operations Research, 4, part 2 ed., vol. 58, pp. 1235-1246, July (3rd Quarter/Summer) 2010


An optimum of a Markov decision process (MDP) is {\it myopic} if it can be obtained by solving a series of static problems. Myopic optima are desirable because they can be computed relatively easily. We identify new classes of MDPs with myopic optima and sequential games with myopic equilibrium points. In one of the classes, the single-period reward is homogeneous with respect to the state variable. We illustrate the results with models of revenue management and investment.