STAT 4382: Homework 3

Due by September 28, 2020

Yuly Koshevnik

First Name Last Name

Do Not Write Below This Line, PLEASE! Problem 1 2 3 4 5 6 7 8 Total

Points

Maximum 15 10 15 10 15 10 15 10 100

Please submit the title page with your homework!

1

I need help on questions 2,4,6 and 7

Problem 1 [15 points = 5 + 5 + 5]

Consider a Markov chain associated with FOUR umbrellas, instead of similar to three, which was consid- ered in the notes. Assume that rains occur with the same probability, p, during the day, and this occurrence does not depend on the process, X = {X(n) : n ≥ 0}

1. Show transition probabilities for the state space S = {1, 2, 3, 4}.

2. Derive stationary distribution for the number of umbrellas.

3. Evaluate limiting expectation, lim n→∞

E [X(n)]

Solution

2

Problem 2 [10 points = 5 + 5]

Proceed with FOUR umbrellas from Problem 1. In addition, assume that the constant probability of rain is p = 0.5.

1. Suppose that initial distribution of X(0) is stationary, determine E [X(0)]

2. Under the limiting distribution of X(n), find the chance that an umbrella is carried at the time n.

Solution

3

Problem 3 [15 points = 5 + 5 + 5]

Consider a Markov chain with SIX states,

S = {0, 1, 2, 3, 4, 5}

and transition probabilities defined as follows:

P0,0 = 1

3 , P0,1 = 0, P0.2 =

1

3 , P0,3 = P0.4 = 0, = P0.5 =

1

3

P1,0 = 1

2 , P1,1 = P1,2 =

1

4 , P1,3 = P1,4 = P1,5 = 0

P2,0 = P2,1 = P2,2 = P2,3 = 0, P2,4 = 1, P2,5 = 0

P3,0 = P3,1 = P3,2 = 1

4 , P3,3 = P3,4 = 0 P3,5 =

1

4

P4,0 = P4,1 = 0, P4,2 = 1, P4,3 = P4,4 = P4,5 = 0

P5,0 = P5,1 = P5,2 = P5,3 = P5,4 = 0, P5,5 = 1

1. Find all transient states

2. Determine recurrent states

3. Which states are absorbing?

Solution

4

Problem 4 [10 points]

Proceed with the transition probabilities described in Problem 3. Determine periodic states with period d > 1

Solution

5

Problem 5 [15 points = 5 + 10]

Consider a Markov chain with the state space S = {0, 1, 2, 3} and transition probabilities as follows:

P0,1 = P0,3 = 1

2 and P0.0 = P0.2 = 0

P1,2 = 1 and P1,0 = P1,1 = P1,3 = 0

P2,0 = P2,1 = P2,2 = 0 and P2,3 = 1

P3,0 = P3,3 = 1

2 and P3,1 = P3,2 = 0

1. Which states are recurrent and which of them are transient?

2. Determine the limiting matrix, limn→∞ P n

Solution

6

Problem 6 [10 points = 6 + 4]

Consider a Markov chain, X = {X(n) : n ≥ 0}, describing three types of a failure of some device. There are FOUR states, S = {0, 1, 2, 3}, where 0 means normal functioning, while three other states indicate the failure type. Transition probability matrix, P, is given as follows:

0 1 2 3

0 a0 a1 a2 a3

1 b1 1− b1 0 0

2 b2 0 1− b2 0

3 b3 0 0 1− b3

Notice that only one type of a failure can occur within a time unit, and values bi determine the probability of restoration within the next unit. All values ai and bi are such that 0 < ai < 1 and 0 < bi < 1.

1. Find limn→∞ P n

2. In a long run, determine the proportion of time this device will be functioning.

Solution

7

Problem 7 [15 points = 5 + 5 + 5]

Consider a Markov chain with the state space, S = {0, 1} and transition probabilities

P0,0 = 1− a and P0,1 = a

P1,0 = b, and P1,1 = 1− b,

where 0 < a < 1 and 0 < b < 1. Consider a random variable, M, the first return time to the initial state X(0) = 0.

1. Find P [M = 1|X(0) = 0]

2. For any m ≥ 2, derive P [M = m], which is equivalent to

P

 ⋂ k≤m−1

[X(k) = 1] ⋂

(X(m) = 0)|X(0) = 0

 3. Derive E [M |X(0) = 0].

Notice that the straightforward calculation leads to

E [M |X(0) = 0] =

Solution

8

Problem 8 [10 points]

Consider a periodic Markov chain with the state space S = {0, 1, 2, 3} and transition probability matrix

0 1 2 3

0 1− p p 0 0

1 1− p 0 p 0

2 1− p 0 0 p

3 1 0 0 0

Determine the limiting probabilities:

lim n→∞

P [X(n) = j|X(0) = i]

for all (i, j)

Solution

9