Homogeneous Markov Decision Processes



Operations Research, 4, part 2 ed., vol. 58, pp. 1235-1246, July (3rd Quarter/Summer) 2010


An optimum of a Markov decision process (MDP) is {\it myopic} if it can be obtained by solving a series of static problems. Myopic optima are desirable because they can be computed relatively easily. We identify new classes of MDPs with myopic optima and sequential games with myopic equilibrium points. In one of the classes, the single-period reward is homogeneous with respect to the state variable. We illustrate the results with models of revenue management and investment.

Matthew Sobel

Social Media
Weatherhead School of Management
Case Western Reserve University

10900 Euclid Avenue
Cleveland, Ohio 44106-7235 USA