Finite Markov Chains

AARW absorbing chain absorption APRW assume chain theory chain with transition Chapter column vector components compute consider covariance matrix denote depend entries equivalence class equivalence relation ergodic chain Example 3a expanded process find the mean finite fixed probability vector fixed vector function fundamental matrix given gives Hence independent trials process industry initial vector j-th Land of Oz large number limiting covariances limiting matrix limiting variance lumpable with respect lumped chain lumped process Markov process Markov property matrix for Example mean and variance mean first passage mean number number of steps obtain original chain original process outcome Oz example partition passage matrix possible process starts PROOF random walk regular chains regular Markov chain s₁ sequence starting vector stimulus elements subset symmetric matrix THEOREM transient transition matrix transition probabilities weak lumpability