Markov Processes and Controlled Markov Chains

  • Zhenting Hou
  • Jerzy A. Filar
  • Anyue Chen

Table of contents

  1. Front Matter
    Pages i-x
  2. Markov Processes

    1. Front Matter
      Pages 1-1
    2. Hanjun Zhang, Qixiang Mei, Xiang Lin, Zhenting Hou
      Pages 57-67
    3. Zhenting Hou, Zaiming Liu, Jiezhong Zou, Xuerong Chen
      Pages 69-92
  3. Controlled Markov Chains and Decision Processes

    1. Front Matter
      Pages 109-109
    2. Seiichi Iwamoto, Takayuki Ueno, Toshiharu Fujita
      Pages 135-149
    3. L. C. M. Kallenberg
      Pages 151-165
    4. Xianping Guo, Weiping Zhu
      Pages 167-188
    5. Masami Kurano, Masami Yasuda, Jun-ichi Nakagami
      Pages 223-232
    6. Jerzy A. Filar, Xianping Guo
      Pages 245-254
    7. A. A. Yushkevich
      Pages 255-286
    8. Sara Pasquali, Wolfgang J. Runggaldier
      Pages 287-302
  4. Stochastic Processes and Martingales

  5. Applications to Finance, Control Systems and Other Related Fields

    1. Front Matter
      Pages 369-369
    2. Zhaojun Yang, Jiezhong Zou
      Pages 389-395
    3. Chao Chen, Jiezhong Zou, Zhenting Hou
      Pages 397-404
    4. Yuanlie Lin, Jerzy A. Filar, Ke Liu
      Pages 405-424
    5. Marta S. Mendiondo, Richard H. Stockbridge
      Pages 425-441
    6. Junping Li, Zhenting Hou, Zaiming Liu, Weiguo Zhang
      Pages 465-474

About this book


The general theory of stochastic processes and the more specialized theory of Markov processes evolved enormously in the second half of the last century. In parallel, the theory of controlled Markov chains (or Markov decision processes) was being pioneered by control engineers and operations researchers. Researchers in Markov processes and controlled Markov chains have been, for a long time, aware of the synergies between these two subject areas. However, this may be the first volume dedicated to highlighting these synergies and, almost certainly, it is the first volume that emphasizes the contributions of the vibrant and growing Chinese school of probability. The chapters that appear in this book reflect both the maturity and the vitality of modern day Markov processes and controlled Markov chains. They also will provide an opportunity to trace the connections that have emerged between the work done by members of the Chinese school of probability and the work done by the European, US, Central and South American and Asian scholars.

Editors and affiliations

  • Zhenting Hou
    • 1
  • Jerzy A. Filar
    • 2
  • Anyue Chen
    • 3
  1. 1.Research DepartmentChangsha Railway UniversityChangshaChina
  2. 2.School of MathematicsUniversity of South AustraliaMawson LakesAustralia
  3. 3.School of Computing and Mathematical SciencesUniversity of GreenwichLondonUK

Bibliographic information

  • DOI
  • Copyright Information Springer-Verlag US 2002
  • Publisher Name Springer, Boston, MA
  • eBook Packages Springer Book Archive
  • Print ISBN 978-1-4613-7968-3
  • Online ISBN 978-1-4613-0265-0
  • Buy this book on publisher's site