USA: +1-585-535-1023

UK: +44-208-133-5697

AUS: +61-280-07-5697

First Entrance Probability

Let                               fij(n)= Probability of arriving atj at time n for the first time, given that the process starts at i

                                    = P [ Xn = j, Xn-1 ≠ j, Xn-2 ≠ j. … X1 ≠ j| X0 = i]



Tij =Min {n : Xn = j | X0 = i}. Then

fij(1)= = Pij (gives the diagonal of the transition matrix)


We can prove the following result :

A state i of a Markov chain is said to be transient if fii < 1 and recurrent if fii = 1.

Also. the mean recurrence time for




1. Test whether the following Markov chains are periodic or aperiodic.


(b)                                                       To

2. Test whether the Markov chain having the following transition matrix is regular and ergodic.

3. Consider the three state Markov chain with transition probability matrix

Prove that the chain is irreducible

4. Find the mean recurrence time for each state of the following Markov chain


1. (a) Periodic, (b) Aperiodic

2. Ergodic but not regular

4. µ00=3.9604, µ11=2.5381, µ12=2.8289.