This work concerns Markov chains evolving on a denumerable sate space, which is endowed with a non-negative reward function with finite support. In this context, the problem of determining the Varadhan function, given by the exponential growth rate of the aggregated rewards, is studied. The main results in this direction are expressed in terms of the idea of asystem of local Poisson equations, and can be summarized as follows: (i) the Varadhan function is determined by one of such systems, and (ii) if a finite set is accessible form any state, then a system of local Poisson equations exits.
A.Alanís-Durán and R.Cavazos-Cadena, An optimality system for finite average Markov decision chains under risk-aversion, Kybernetika48 (2012), 83–104.
2.
A.Biswas, V.S.Borkar and K.Suresh Kumar, Risk-sensitive control with near monotone cost, Appl. Math. Opt.62 (2009), 145–163.
3.
V.S.Borkar and K.Suresh Kumar, Singular perturbations in risk-sensitive stochastic control, SIAM J. Control Optim.48 (2010), 3675–3697.
4.
R.Cavazos-Cadena, The risk-sensitive Poisson equation for a communicating Markov chain on a denumerable state space, Kybernetika45 (2009), 716–736.
5.
R.Cavazos-Cadena and D.Hernández-Hernández, A characterization of exponential functionals in finite Markov chains, Math. Methods Oper. Res.60 (2004), 399–414.
6.
R.Cavazos-Cadena and D.Hernández-Hernández, A system of Poisson equations for a non-constant Varadhan functional on a finite state space, Appl. Math. Opt.53 (2006), 101–119.
7.
A.Dembo and O.Zeitouni, Large Deviations Techniques and Applications, Jones and Bartlett, Boston, MA, 1993.
8.
G.B.Di Masi and L.Stettner, Risk-sensitive control of discrete time Markov processes with infinite horizon, SIAM J. Control Optim.38 (1999), 61–78.
9.
G.B.Di Masi and L.Stettner, Infinite horizon risk sensitive control of discrete time Markov processes with small risk, Syst. Control Lett.40 (2000), 15–20.
10.
G.B.Di Masi and L.Stettner, Infinite horizon risk sensitive control of discrete time Markov processes under minorization property, SIAM J. Control Optim.46 (2007), 231–252.
11.
M.D.Donsker and S.R.Varadhan, Asymptotic evaluation of certain Markov process expectations for large time, I, Commun. Pur. Appl. Math.28 (1975), 1–47.
12.
M.D.Donsker and S.R.Varadhan, Asymptotic evaluation of certain Markov process expectations for large time, II, Commun. Pur. Appl. Math.28 (1975), 279–301.
13.
M.D.Donsker and S.R.Varadhan, Asymptotic evaluation of certain Markov process expectations for large time, III, Commun. Pur. Appl. Math.29 (1975), 389–461.
14.
W.H.Fleming and W.M.McEneaney, Risk sensitive control on an infinite horizon, SIAM J. Control Optim.33 (1995), 1881–1915.
15.
W.H.Fleming and S.J.Sheu, Asymptotics of the principal eigenvalue and eigenfunction of a nearly first-order operator with large potential, Ann. Probab.25 (1997), 1953–1994.
16.
D.Hernández-Hernández and S.I.Marcus, Existence of risk sensitive optimal stationary polices for controlled Markov processes, Appl. Math. Opt.40 (1999), 273–285.
N.Ichihara, Phase transitions for controlled Markov chains on infinite graphs, Preprint, 2014.
19.
A.Jaśkiewicz, Average optimality for risk sensitive control with general state space, Ann. Appl. Probab.17 (2007), 654–675.
20.
H.Kaise and H.Nagai, Ergodic type Bellman equations of risk-sensitive control with large parameters and their singular limits, Asymptotic Analysis20 (1999), 279–299.
21.
I.Kontoyiannis and S.P.Meyn, Spectral theory and limit theorems for geometrically ergodic Markov processes, Ann. Appl. Probab.13 (2003), 304–362.
22.
C.D.Meyer, Matrix Analysis and Applied Linear Algebra, SIAM, Philadelphia, 2000.
23.
M.L.Puterman, Markov Decision Processes: Discrete Stochastic Dynamic Programming, Wiley, New York, 2005.
