Finite Markov Chains

AARW absorbing chain absorption APRW assume chain theory chain with transition Chapter column vector components compute condition for lumpability consider covariance matrix denote depend entries equivalence class ergodic chain ergodic set Example 3a expanded process find the mean finite fixed probability vector fixed vector function fundamental matrix given gives Hence independent trials process industry initial vector j-th Land of Oz large number limiting covariances limiting matrix limiting variance lumpable with respect lumped chain lumped process Markov process Markov property mean and variance mean first passage mean number number of steps obtain original chain original process outcome Oz example partition passage matrix process starts PROOF random walk regular chains regular Markov chain s₁ sequence starting vector stimulus elements subset symmetric matrix THEOREM transient transition matrix transition probabilities weak lumpability