Mathematics

Simone

5 January, 06:42

In Markov chains, if I was given a transition probability matrix with each of the probabilities specified, then how do I determine the following:1 - Probability that state y is visited at least n times given that you start in state x. I know that I can solve it using Px (# of visits to state y ≥ n) = rhoxy (rhoyy) n-1 where rhoxy is the probability that starting at state x, I will be in state y in some positive time (i. e. rhoxy=Px (Ty<[infinity])). But I am not sure how to calculate rhoxy and I have spent so much time trying to figure it out!