On time reversal of piecewise deterministic Markov processes

Andreas Löpker (Helmut Schmidt University, Hamburg)
Zbigniew Palmowski (University of Wroclaw)

Abstract


We study the time reversal of a general Piecewise Deterministic Markov Process (PDMP). The time reversed process is defined as $X_{(T-t)-}$, where $T$ is some given time and $X_t$ is a stationary PDMP. We obtain the parameters of the reversed process, like the jump intensity and the jump measure.

Full Text: Download PDF | View PDF online (requires PDF plugin)

Pages: 1-29

Publication Date: January 23, 2013

DOI: 10.1214/EJP.v18-1958

References

  • Albrecher, Hansjörg; Thonhauser, Stefan. Optimality results for dividend problems in insurance. Rev. R. Acad. Cienc. Exactas Fís. Nat. Ser. A Math. RACSAM 103 (2009), no. 2, 295--320. MR2582635
  • E. Altman, K. Avrachenkov, A. Kherani, and B. Prabhu, phPerformance Analysis and Stochastic Stability of Congestion Control Protocols, Tech. Report RR-5262, INRIA, Sophia-Antipolis, France, July 2004.
  • Anick, D.; Mitra, D.; Sondhi, M. M. Stochastic theory of a data-handling system with multiple sources. Bell System Tech. J. 61 (1982), no. 8, 1871--1894. MR0685312
  • Asmussen, Søren. Stationary distributions for fluid flow models with or without Brownian noise. Comm. Statist. Stochastic Models 11 (1995), no. 1, 21--49. MR1316767
  • Asmussen, Søren. Applied probability and queues. Second edition. Applications of Mathematics (New York), 51. Stochastic Modelling and Applied Probability. Springer-Verlag, New York, 2003. xii+438 pp. ISBN: 0-387-00211-1 MR1978607
  • Baccelli, Francois; Kim, Ki Baek; McDonald, David R. Equilibria of a class of transport equations arising in congestion control. Queueing Syst. 55 (2007), no. 1, 1--8. MR2293562
  • Bar-Lev, Shaul K.; Parlar, Mahmut; Perry, David. On the EOQ model with inventory-level-dependent demand rate and random yield. Oper. Res. Lett. 16 (1994), no. 3, 167--176. MR1306220
  • J.-B. Bardet, A. Christen, A. Guillin, F. Malrieu, and P.-A. Zitt, phTotal variation estimates for the TCP process, ArXiv e-prints (2011).
  • Bekker, R.; Borst, S. C.; Boxma, O. J.; Kella, O. Queues with workload-dependent arrival and service rates. Queueing Syst. 46 (2004), no. 3-4, 537--556. MR2068140
  • Billingsley, Patrick. Convergence of probability measures. John Wiley & Sons, Inc., New York-London-Sydney 1968 xii+253 pp. MR0233396
  • Borovkov, K.; Novikov, A. On a piece-wise deterministic Markov process model. Statist. Probab. Lett. 53 (2001), no. 4, 421--428. MR1856167
  • Borovkov, K.; Vere-Jones, D. Explicit formulae for stationary distributions of stress release processes. J. Appl. Probab. 37 (2000), no. 2, 315--321. MR1780992
  • Boxma, Onno; Kaspi, Haya; Kella, Offer; Perry, David. On/off storage systems with state-dependent input, output, and switching rates. Probab. Engrg. Inform. Sci. 19 (2005), no. 1, 1--14. MR2104547
  • Boxma, Onno; Perry, David; Stadje, Wolfgang; Zacks, Shelemyahu. A Markovian growth-collapse model. Adv. in Appl. Probab. 38 (2006), no. 1, 221--243. MR2213972
  • Brockwell, P. J. Stationary distributions for dams with additive input and content-dependent release rate. Advances in Appl. Probability 9 (1977), no. 3, 645--663. MR0478405
  • Browne, Sid; Sigman, Karl. Work-modulated queues with applications to storage processes. J. Appl. Probab. 29 (1992), no. 3, 699--712. MR1174444
  • Çinlar, E.; Pinsky, M. A stochastic integral in storage theory. Z. Wahrscheinlichkeitstheorie und Verw. Gebiete 17 (1971), 227--240. MR0292194
  • Chafaï, Djalil; Malrieu, Florent; Paroux, Katy. On the long time behavior of the TCP window size process. Stochastic Process. Appl. 120 (2010), no. 8, 1518--1534. MR2653264
  • Chung, K. L.; Walsh, John B. To reverse a Markov process. Acta Math. 123 1969 225--251. MR0258114
  • Costa, O. L. V. Stationary distributions for piecewise-deterministic Markov processes. J. Appl. Probab. 27 (1990), no. 1, 60--73. MR1039184
  • Dassios, A.; Embrechts, P. Martingales and insurance risk. Comm. Statist. Stochastic Models 5 (1989), no. 2, 181--217. MR1000630
  • Dassios, Angelos; Jang, Jiwook. The distribution of the interval between events of a Cox process with shot noise intensity. J. Appl. Math. Stoch. Anal. 2008, Art. ID 367170, 14 pp. MR2461288
  • Davis, M. H. A. Piecewise-deterministic Markov processes: a general class of nondiffusion stochastic models. With discussion. J. Roy. Statist. Soc. Ser. B 46 (1984), no. 3, 353--388. MR0790622
  • Davis, M. H. A. Markov models and optimization. Monographs on Statistics and Applied Probability, 49. Chapman & Hall, London, 1993. xiv+295 pp. ISBN: 0-412-31410-X MR1283589
  • Dufour, François; Costa, Oswaldo L. V. Stability of piecewise-deterministic Markov processes. SIAM J. Control Optim. 37 (1999), no. 5, 1483--1502 (electronic). MR1710229
  • Dumas, Vincent; Guillemin, Fabrice; Robert, Philippe. A Markovian analysis of additive-increase multiplicative-decrease algorithms. Adv. in Appl. Probab. 34 (2002), no. 1, 85--111. MR1895332
  • Embrechts, Paul; Schmidli, Hanspeter. Ruin estimation for a general insurance risk model. Adv. in Appl. Probab. 26 (1994), no. 2, 404--422. MR1272719
  • Faggionato, A.; Gabrielli, D.; Ribezzi Crivellari, M. Non-equilibrium thermodynamics of piecewise deterministic Markov processes. J. Stat. Phys. 137 (2009), no. 2, 259--304. MR2559431
  • Hansen, Lars Peter; Scheinkman, José Alexandre. Back to the future: generating moment implications for continuous-time Markov processes. Econometrica 63 (1995), no. 4, 767--804. MR1343081
  • Harrison, J. Michael; Resnick, Sidney I. The stationary distribution and first exit probabilities of a storage process with general release rule. Math. Oper. Res. 1 (1976), no. 4, 347--358. MR0461726
  • Harrison, J. Michael; Resnick, Sidney I. The recurrence classification of risk and storage processes. Math. Oper. Res. 3 (1978), no. 1, 57--66. MR0496972
  • P.G. Harrison and N. Thomas, phProduct-form solution in PEPA via the reversed process, Network Performance Engineering (D. Kouvatsos, ed.), Lecture Notes in Computer Science, vol. 5233, Springer Berlin Heidelberg, 2011, pp. 343--356.
  • Hartman, Philip. Ordinary differential equations. Reprint of the second edition. Birkhäuser, Boston, Mass., 1982. xv+612 pp. ISBN: 3-7643-3068-6 MR0658490
  • Jacobsen, Martin. Point process theory and applications. Marked point and piecewise deterministic processes. Probability and its Applications. Birkhäuser Boston, Inc., Boston, MA, 2006. xii+328 pp. ISBN: 978-0-8176-4215-0; 0-8176-4215-3 MR2189574
  • Jacod, Jean; Protter, Philip. Time reversal on Lévy processes. Ann. Probab. 16 (1988), no. 2, 620--641. MR0929066
  • Jacod, J.; Skorohod, A. V. Jumping filtrations and martingales with finite variation. Séminaire de Probabilités, XXVIII, 21--35, Lecture Notes in Math., 1583, Springer, Berlin, 1994. MR1329098
  • Kelly, Frank P. Reversibility and stochastic networks. Wiley Series in Probability and Mathematical Statistics. John Wiley & Sons, Ltd., Chichester, 1979. viii+230 pp. ISBN: 0-471-27601-4 MR0554920
  • Kolmogoroff, A. Zur Theorie der Markoffschen Ketten. (German) Math. Ann. 112 (1936), no. 1, 155--160. MR1513044
  • bysame, phZur Umkehrbarkeit der statistischen Naturgesetze., Math. Ann. 113 (1936), 766--772 (German).
  • Last, Günter. Ergodicity properties of stress release, repairable system and workload models. Adv. in Appl. Probab. 36 (2004), no. 2, 471--498. MR2058146
  • Borovkov, K.; Last, G. On level crossings for a general class of piecewise-deterministic Markov processes. Adv. in Appl. Probab. 40 (2008), no. 3, 815--834. MR2454034
  • Last, Günter; Szekli, Ryszard. Stochastic comparison of repairable systems by coupling. J. Appl. Probab. 35 (1998), no. 2, 348--370. MR1641801
  • Löpker, Andreas; Stadje, Wolfgang. Hitting times and the running maximum of Markovian growth-collapse processes. J. Appl. Probab. 48 (2011), no. 2, 295--312. MR2840300
  • Maulik, Krishanu; Zwart, Bert. Tail asymptotics for exponential functionals of Lévy processes. Stochastic Process. Appl. 116 (2006), no. 2, 156--177. MR2197972
  • M. Miyazawa, phReversibility in Queueing Models, ArXiv e-prints (2012).
  • Nagasawa, Masao. Time reversions of Markov processes. Nagoya Math. J. 24 1964 177--204. MR0169290
  • Nagasawa, Masao. Time reversal of Markov processes and relativistic quantum theory. Chaos Solitons Fractals 8 (1997), no. 11, 1711--1772. MR1477258
  • Norris, J. R. Markov chains. Reprint of 1997 original. Cambridge Series in Statistical and Probabilistic Mathematics, 2. Cambridge University Press, Cambridge, 1998. xvi+237 pp. ISBN: 0-521-48181-3 MR1600720
  • Ogata, Y.; Vere-Jones, D. Inference for earthquake models: a self-correcting model. Stochastic Process. Appl. 17 (1984), no. 2, 337--347. MR0751210
  • Palmowski, Zbigniew; Rolski, Tomasz. A technique for exponential change of measure for Markov processes. Bernoulli 8 (2002), no. 6, 767--785. MR1963661
  • Rolski, Tomasz; Schmidli, Hanspeter; Schmidt, Volker; Teugels, Jozef. Stochastic processes for insurance and finance. Wiley Series in Probability and Statistics. John Wiley & Sons, Ltd., Chichester, 1999. xviii+654 pp. ISBN: 0-471-95925-1 MR1680267
  • M. Schäl, phSprungprozesse, Lecture Notes, University of Bonn (1997) (English).
  • Schäl, Manfred. On piecewise deterministic Markov control processes: control of jumps and of risk processes in insurance. The interplay between insurance, finance and control (Aarhus, 1997). Insurance Math. Econom. 22 (1998), no. 1, 75--91. MR1625803
  • E. Schrödinger, phÜber die Umkehrung der Naturgesetze., (1931) (German).
  • Tanaka, Hiroshi. Time reversal of random walks in one-dimension. Tokyo J. Math. 12 (1989), no. 1, 159--174. MR1001739
  • E.A. Van~Doorn and W.R.W. Scheinhardt, phAnalysis of birth-death fluid queues, Memorandum, University of Twente (1996).
  • Löpker, Andreas H.; van Leeuwaarden, Johan S. H. Transient moments of the TCP window size process. J. Appl. Probab. 45 (2008), no. 1, 163--175. MR2409318
  • van Leeuwaarden, J. S. H.; Löpker, A. H.; Ott, T. J. TCP and iso-stationary transformations. Queueing Syst. 63 (2009), no. 1-4, 459--475. MR2576022
  • D. Vere-Jones, phOn the variance properties of stress release models., Austral. J. Statist. 30A (1988), 123--135.
  • Walsh, John B. Time reversal and the completion of Markov processes. Invent. Math. 10 1970 57--81. MR0270441
  • Weiss, Gideon. Time-reversibility of linear stochastic processes. J. Appl. Probability 12 (1975), no. 4, 831--836. MR0385998
  • Wobst, Reinhard. On jump processes with drift. Dissertationes Math. (Rozprawy Mat.) 202 (1983), 51 pp. MR0711663
  • Yeh, J. Lectures on real analysis. World Scientific Publishing Co., Inc., River Edge, NJ, 2000. xvi+548 pp. ISBN: 981-02-3936-X MR1779377
  • Zheng, Xiao Gu. Ergodic theorems for stress release processes. Stochastic Process. Appl. 37 (1991), no. 2, 239--258. MR1102872
Bookmark and Share


Creative Commons License
This work is licensed under a Creative Commons Attribution 3.0 License.