These are used to compute n-step transition probabilities as given by

for all *i , j, n *and 0 ≥ *v *≥ *n.*

When *v *= 1, then

for all *i, j, n*

for all *i, j, n*

The above equations show that the n-step transition probabilities can be obtained from the one step transition probabilities recursively.

Let *n *= 2, then we can write

for all *i, j, n*

The above equations show that the n-step transition probabilities can be obtained from the one step transition probabilities recursively.

Let *n *= 2, then we can write