Temporal analysis of hidden and standard Markov processes with final sequences of states [Articol]

Abstract

This research is focused on temporal analysis of hidden and standard Markov processes with final sequences of states and the related games. Initially, the stochastic systems with final sequences of states are considered and enhanced methods for determining the distributions of the evolution time and the game duration are proposed. These approaches are extended later for the hidden Markov models with final sequences of observable states. Additionally, the definition of the total alert time is generalized for the hidden and standard Markov processes and the developed numerical algorithms for the evolution time are adapted to cover also the computation of distribution of the total alert time. For both, hidden and standard Markov processes, and for the associated games, it is shown that the evolution time, the total alert time and the game duration have homogeneous linear recurrent distributions, which allows us to easily evaluate their main probabilistic characteristics, using the previously developed polynomial algorithms.