Continuous Time HMM

  • William Turin
Part of the Information Technology: Transmission, Processing and Storage book series (PSTE)


In the previous chapters, we considered HMMs which are probabilistic functions of discrete Markov chains. In the continuous time HMM, observations depend on states of a continuous time Markov process. The continuous time models assume that during a small time interval the probability of a state change is also small. Therefore, if the time interval is small, the state transition probability matrixPis close to a unit matrixI.When we estimated the model parameters on the basis of experimental data in Sec. 1.5, we discovered that the diagonal elements of the transition probability matrices are close to one. For that reason, it was convenient to represent the models with the matricesR = P — I.The other consideration which must be taken into account is the accuracy of all the computations: we would rather express all the equations containingPin terms ofRto avoid computations with numbers having different precision. These and other considerations suggest that we might make another step and consider a limiting case of an HMM when the time unit becomes infinitesimally small. In the limit we obtain a continuous time HMM.


Transition Probability Matrix Interarrival Time Continuous Time Model Discrete Time Markov Chain Single Server Queue 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    J. Abate and W. Whitt, “The Fourier-series method for inverting transforms of probability distributions,” Queueing Syst. 10 5–88, (1992).MathSciNetzbMATHCrossRefGoogle Scholar
  2. 2.
    S. Asmussen and O. Nerman, “Fitting phase-type distributions via the EM algorithm,” Symposium i Anvendt Statistik Copenhagen, 335–346, (1991).Google Scholar
  3. 3.
    W. Fisher and K. Meier-Hellstem, “The Markov-modulated Poisson process (MMPP) cookbook,” Performance Evaluation 18 149–171, (1992).CrossRefGoogle Scholar
  4. 4.
    F. R. Gantmacher The Theory of Matrices (Chelsea Publishing Co., New York, 1959).zbMATHGoogle Scholar
  5. 5.
    H. Heffes and D. M. Lucantoni, “A Markov modulated characterization of packetized voice and data traffic and related statistical multiplexer performance,”IEEE J. on Selected Areas in Comm. SAC-4 (6)856–868, Sept. 1986.CrossRefGoogle Scholar
  6. 6.
    D. M. Lucantoni An Algorithmic Analysis of a Communication Model with Retransmissions of Flawed Messages (Pitman, London, 1983).Google Scholar
  7. 7.
    D. M. Lucantoni, K. S. Meier-Hellstern, and M. F. Neuts, “A single server queue with server vacations and a class of non-renewal arrival processes,” Adv. in Appl. Probability 22 (3), 676–705, Sep. (1990).MathSciNetzbMATHCrossRefGoogle Scholar
  8. 8.
    D. M. Lucantoni, “New results on the single server queue with a batch Markovian arrival processes,” Comm. Statist. Stochastic Models 7, 1–46, (1991).MathSciNetzbMATHCrossRefGoogle Scholar
  9. 9.
    K. S. Meier, “A fitting algorithm for Markov-modulated Poisson processes having two arrival rates,” European J. Oper. Res. 29 370–377, (1987).MathSciNetzbMATHCrossRefGoogle Scholar
  10. 10.
    C. Moler and C. van Loan, “Nineteen dubious ways to compute the exponential of a matrix,” SIAM Rev. 20 801–836, (1978).MathSciNetzbMATHCrossRefGoogle Scholar
  11. 11.
    M.F. Neuts Matrix-Geometric Solutions in Stochastic Models (Johns Hopkins University Press, Baltimore, 1981).zbMATHGoogle Scholar
  12. 12.
    M. F. Neuts, “A versatile Markovian point process,” J. Appl. Probab. 16 764–779 (1979).MathSciNetzbMATHCrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media New York 2004

Authors and Affiliations

  • William Turin
    • 1
  1. 1.AT&T Labs—ResearchFlorham ParkNew JerseyUSA

Personalised recommendations