A note on linearization methods and dynamic programming principles for stochastic discontinuous control problems

Dan Goreac (Université Paris-Est Marne-la-Vallée)
Oana Silvia Serea (Université de Perpignan)

Abstract


Using the linear programming approach to stochastic control introduced by Buckdahn, Goreac, and Quincampoix, and by Goreac and Serea, we provide a semigroup property for some set of probability measures leading to dynamic programming principles for stochastic control problems. An abstract principle is provided for general bounded costs. Linearized versions are obtained under further (semi)continuity assumptions.

Full Text: Download PDF | View PDF online (requires PDF plugin)

Pages: 1-12

Publication Date: February 27, 2012

DOI: 10.1214/ECP.v17-1844

References

  • Aubin, Jean-Pierre; Cellina, Arrigo. Differential inclusions. Set-valued maps and viability theory. Grundlehren der Mathematischen Wissenschaften [Fundamental Principles of Mathematical Sciences], 264. Springer-Verlag, Berlin, 1984. xiii+342 pp. ISBN: 3-540-13105-1 MR0755330
  • Barron, E. N.; Jensen, R. Semicontinuous viscosity solutions for Hamilton-Jacobi equations with convex Hamiltonians. Comm. Partial Differential Equations 15 (1990), no. 12, 1713--1742. MR1080619
  • Bhatt, Abhay G.; Borkar, Vivek S. Occupation measures for controlled Markov processes: characterization and optimality. Ann. Probab. 24 (1996), no. 3, 1531--1562. MR1411505
  • Borkar, Vivek; Gaitsgory, Vladimir. Averaging of singularly perturbed controlled stochastic differential equations. Appl. Math. Optim. 56 (2007), no. 2, 169--209. MR2352935
  • Bouchard, Bruno; Touzi, Nizar. Weak dynamic programming principle for viscosity solutions. SIAM J. Control Optim. 49 (2011), no. 3, 948--962. MR2806570
  • Buckdahn, R.; Goreac, D.; Quincampoix, M. Stochastic optimal control and linear programming approach. Appl. Math. Optim. 63 (2011), no. 2, 257--276. MR2772196
  • Fleming, Wendell H.; Soner, H. Mete. Controlled Markov processes and viscosity solutions. Second edition. Stochastic Modelling and Applied Probability, 25. Springer, New York, 2006. xviii+429 pp. ISBN: 978-0387-260457; 0-387-26045-5 MR2179357
  • Frankowska, Hélène. Lower semicontinuous solutions of Hamilton-Jacobi-Bellman equations. SIAM J. Control Optim. 31 (1993), no. 1, 257--272. MR1200233
  • Gaitsgory, Vladimir; Quincampoix, Marc. Linear programming approach to deterministic infinite horizon optimal control problems with discounting. SIAM J. Control Optim. 48 (2009), no. 4, 2480--2512. MR2556353
  • Goreac, Dan; Serea, Oana-Silvia. Mayer and optimal stopping stochastic control problems with discontinuous cost. J. Math. Anal. Appl. 380 (2011), no. 1, 327--342. MR2786205
  • Kurtz, Thomas G.; Stockbridge, Richard H. Existence of Markov controls and characterization of optimal Markov controls. SIAM J. Control Optim. 36 (1998), no. 2, 609--653 (electronic). MR1616514
  • Lasserre, Jean B.; Henrion, Didier; Prieur, Christophe; Trélat, Emmanuel. Nonlinear optimal control via occupation measures and LMI-relaxations. SIAM J. Control Optim. 47 (2008), no. 4, 1643--1666. MR2421324
  • Stockbridge, Richard H. Time-average control of martingale problems: existence of a stationary solution. Ann. Probab. 18 (1990), no. 1, 190--205. MR1043943
  • Yin, G. George; Zhang, Qing. Continuous-time Markov chains and applications. A singular perturbation approach. Applications of Mathematics (New York), 37. Springer-Verlag, New York, 1998. xvi+349 pp. ISBN: 0-387-98244-2 MR1488963
  • Yong, Jiongmin; Zhou, Xun Yu. Stochastic controls. Hamiltonian systems and HJB equations. Applications of Mathematics (New York), 43. Springer-Verlag, New York, 1999. xxii+438 pp. ISBN: 0-387-98723-1 MR1696772
Bookmark and Share


Creative Commons License
This work is licensed under a Creative Commons Attribution 3.0 License.