<aside> 🚧

Under construction. This page outlines strategies for acing the final exam.

</aside>

Question 1 (30% of the total points)

Given a Markov chain, analyze its properties.

• Find communicating classes. (Lecture 14)
• Find recurrent/transient states. (Lecture 15)
• Find absorption probabilities (Lecture 15)
• Find mean hitting times (Lecture 15)
• Find mean return times (Lecture 15)

This problem will be “mechanical”. Study lecture notes.

Question 2 (20% of the total points)

Application of Markov chains. (Lecture 13, 14)

This question will involve defining Markov chain relevant to the application. Possibly transform Markov chain to find quantities of interest.

Solve practice problems.

Question 3 (30% of the total points)

Given a Markov Decision Processes, analyze its properties.

Question 4 (20% of the total points)

Applications of Markov Decision Processes (future lectures)