USA: +1-585-535-1023

UK: +44-208-133-5697

AUS: +61-280-07-5697

First Entrance Probability

Let                               fij(n)= Probability of arriving atj at time n for the first time, given that the process starts at i

                                    = P [ Xn = j, Xn-1 ≠ j, Xn-2 ≠ j. … X1 ≠ j| X0 = i]

 

Let

Tij =Min {n : Xn = j | X0 = i}. Then

fij(1)= = Pij (gives the diagonal of the transition matrix)

and

We can prove the following result :

A state i of a Markov chain is said to be transient if fii < 1 and recurrent if fii = 1.

Also. the mean recurrence time for

 

PROBLEMS

 

1. Test whether the following Markov chains are periodic or aperiodic.

To

(b)                                                       To

2. Test whether the Markov chain having the following transition matrix is regular and ergodic.

3. Consider the three state Markov chain with transition probability matrix

Prove that the chain is irreducible

4. Find the mean recurrence time for each state of the following Markov chain

ANSWERS

1. (a) Periodic, (b) Aperiodic

2. Ergodic but not regular

4. µ00=3.9604, µ11=2.5381, µ12=2.8289.