A Gaussian process approximation for two-color randomly reinforced urns

Lixin Zhang (Zhejiang University)

Abstract


The Polya urn has been extensively studied and is widely applied in many disciplines. An important application  is to use urn models to develop randomized treatment allocation schemes in clinical studies. The randomly reinforced urn was recently proposed. In this paper, we prove a Gaussian process approximation for the sequence of random composotions of a two-color randomly reinforced urn for both the cases with the equal and unequal reinforcement means. The Gaussian process is a tail stochastic integral with respect to  a Brownian motion. By using the Gaussian approximation, the law of the iterated logarithm and the functional  central limit theorem in both the stable convergence sense and the almost-sure conditional convergence sense are established. Also as a consequence, we are able to prove that the limit distribution of the normalized urn composition has no points masses both  when the reinforcements means are equal and unequal under the assumption of only finite $(2+\epsilon)$-th moments.

Full Text: Download PDF | View PDF online (requires PDF plugin)

Pages: 1-19

Publication Date: September 18, 2014

DOI: 10.1214/EJP.v19-3432

References

  • Aletti, Giacomo; May, Caterina; Secchi, Piercesare. On the distribution of the limit proportion for a two-color, randomly reinforced urn with equal reinforcement distributions. Adv. in Appl. Probab. 39 (2007), no. 3, 690--707. MR2357377
  • Aletti, Giacomo; May, Caterina; Secchi, Piercesare. A central limit theorem, and related results, for a two-color randomly reinforced urn. Adv. in Appl. Probab. 41 (2009), no. 3, 829--844. MR2571318
  • Bai, Z. D. and Hu, F. (2005). Strong consistency and asymptotic normality for urn models. Ann. Appl. Probab., 12: 914-940.
  • Bai, Z. D.; Hu, Feifang; Rosenberger, William F. Asymptotic properties of adaptive designs for clinical trials with delayed response. Ann. Statist. 30 (2002), no. 1, 122--139. MR1892658
  • Bai, Z. D.; Hu, Feifang; Zhang, Li-Xin. Gaussian approximation theorems for urn models and their applications. Ann. Appl. Probab. 12 (2002), no. 4, 1149--1173. MR1936587
  • Beggs, A. W. On the convergence of reinforcement learning. J. Econom. Theory 122 (2005), no. 1, 1--36. MR2131871
  • Berti, Patrizia; Crimaldi, Irene; Pratelli, Luca; Rigo, Pietro. Central limit theorems for multicolor urns with dominated colors. Stochastic Process. Appl. 120 (2010), no. 8, 1473--1491. MR2653262
  • Berti, Patrizia; Crimaldi, Irene; Pratelli, Luca; Rigo, Pietro. A central limit theorem and its applications to multicolor randomly reinforced urns. J. Appl. Probab. 48 (2011), no. 2, 527--546. MR2840314
  • Chauvin, Brigitte; Pouyanne, Nicolas; Sahnoun, Reda. Limit distributions for large Pólya urns. Ann. Appl. Probab. 21 (2011), no. 1, 1--32. MR2759195
  • Crimaldi, Irene. An almost sure conditional convergence result and an application to a generalized Pólya urn. Int. Math. Forum 4 (2009), no. 21-24, 1139--1156. MR2524635
  • Csörgő, M.; Révész, P. Strong approximations in probability and statistics. Probability and Mathematical Statistics. Academic Press, Inc. [Harcourt Brace Jovanovich, Publishers], New York-London, 1981. 284 pp. ISBN: 0-12-198540-7 MR0666546
  • Durham, S. D.; Flournoy, N.; Li, W. A sequential design for maximizing the probability of a favourable response. Canad. J. Statist. 26 (1998), no. 3, 479--495. MR1646698
  • Durham, S. D. and sc Yu, K. F. (1990). Randomized play-the leader rules for sequential sampling from two populations. Probability in Enginerring and Information Science, 26 (4): 355-367.
  • Eberlein, Ernst. On strong invariance principles under dependence assumptions. Ann. Probab. 14 (1986), no. 1, 260--270. MR0815969
  • Eggenberger, F. and sc Pólya, G. (1923). Uber die Statistik verketteter Vorgänge. Zeitschrift Angew. Math. Mech., 3: 279-289.
  • Erev, I. and sc Roth, A. (1998). Predicting how people play games: reinforcement learning in experimental games with unique, mixed strategy equilibria. Amer. Econ. Rev., 88: 848-881.
  • Hall, P.; Heyde, C. C. Martingale limit theory and its application. Probability and Mathematical Statistics. Academic Press, Inc. [Harcourt Brace Jovanovich, Publishers], New York-London, 1980. xii+308 pp. ISBN: 0-12-319350-8 MR0624435
  • Hanson, D. L.; Russo, Ralph P. Some results on increments of the Wiener process with applications to lag sums of i.i.d. random variables. Ann. Probab. 11 (1983), no. 3, 609--623. MR0704547
  • Hopkins, Ed; Posch, Martin. Attainability of boundary points under reinforcement learning. Games Econom. Behav. 53 (2005), no. 1, 110--125. MR2173863
  • Hu, Feifang; Rosenberger, William F. The theory of response-adaptive randomization in clinical trials. Wiley Series in Probability and Statistics. Wiley-Interscience [John Wiley & Sons], Hoboken, NJ, 2006. xiv+218 pp. ISBN: 978-0-471-65396-7; 0-471-65396-9 MR2245329
  • Hu, Feifang; Zhang, Li-Xin. Asymptotic properties of doubly adaptive biased coin designs for multitreatment clinical trials. Ann. Statist. 32 (2004), no. 1, 268--301. MR2051008
  • Janson, Svante. Functional limit theorems for multitype branching processes and generalized Pólya urns. Stochastic Process. Appl. 110 (2004), no. 2, 177--245. MR2040966
  • Janson, Svante. Limit theorems for triangular urn schemes. Probab. Theory Related Fields 134 (2006), no. 3, 417--452. MR2226887
  • Li, W., sc Durham, S. D. and sc Flournoy, N. (1996). Randomized polya urn designs. Proceedings of the Biometric Section of the Statistical Association: 166-170.
  • May, Caterina; Flournoy, Nancy. Asymptotics in response-adaptive designs generated by a two-color, randomly reinforced urn. Ann. Statist. 37 (2009), no. 2, 1058--1078. MR2502661
  • Martin, C. F.; Ho, Y. C. Value of information in the Polya urn process. Inform. Sci. 147 (2002), no. 1-4, 65--90. MR1940746
  • Monrad, Ditlev; Philipp, Walter. Nearby variables with nearby conditional laws and a strong approximation theorem for Hilbert space valued martingales. Probab. Theory Related Fields 88 (1991), no. 3, 381--404. MR1100898
  • Muliere, Pietro; Paganoni, Anna Maria; Secchi, Piercesare. A randomly reinforced urn. J. Statist. Plann. Inference 136 (2006), no. 6, 1853--1874. MR2255601
  • Muliere, Pietro; Paganoni, Anna Maria; Secchi, Piercesare (2006b). Randomly reinforced urns for clinical trials with continuous responses. In SIS-Proceedings of the XLIII Scientific Meeting, 403- 414. Cleup, Padova.
  • Paganoni, Anna Maria; Secchi, Piercesare. A numerical study for comparing two response-adaptive designs for continuous treatment effects. Stat. Methods Appl. 16 (2007), no. 3, 321--346. MR2413518
  • Pólya, G. (1931). Sur quelques points de la théorie des probabilités. Ann. Inst. Poincaré, 1: 117-161.
  • Zhang, Li-Xin. Strong approximations of martingale vectors and their applications in Markov-chain adaptive designs. Acta Math. Appl. Sin. Engl. Ser. 20 (2004), no. 2, 337--352. MR2064011
  • Zhang, LiXin; Hu, FeiFang. The Gaussian approximation for multi-color generalized Friedman's urn model. Sci. China Ser. A 52 (2009), no. 6, 1305--1326. MR2520576
  • Zhang, Li-Xin; Hu, Feifang; Cheung, Siu Hung. Asymptotic theorems of sequential estimation-adjusted urn models. Ann. Appl. Probab. 16 (2006), no. 1, 340--369. MR2209345
  • Zhang, Li-Xin; Hu, Feifang; Cheung, Siu Hung; Chan, Wai Sum. Immigrated urn models—theoretical properties and applications. Ann. Statist. 39 (2011), no. 1, 643--671. MR2797859
  • Zhang, Li-Xin; Hu, Feifang; Cheung, Siu Hung; Chan, Wai Sum. Asymptotic properties of multicolor randomly reinforced Pólya urns. Adv. in Appl. Probab. 46 (2014), no. 2, 585--602. MR3215547


Creative Commons License
This work is licensed under a Creative Commons Attribution 3.0 License.