Markov Chains and Applications of Matrix Methods: Fixed Point and Absorbing Markov Chains (UMAP)

The Markov process pulls together related prior probabilities into chains of events. The description of the process in matrix form allows one to make long-range predictions. Students define a Markov chain, interpret powers of matrices representing Markov chains, recognize certain processes as Markov chains, formulate a matrix of transition probabilties from a tree diagram of a Markov chain.

Table of Contents:

1. MARKOV CHAINS
1.1 Introduction
1.2 Tree Diagrams
1.3 Calculating Probabilities From a Tree Diagram
1.4 The Matrix Representation of a Markov Chain
1.5 Experiment 1
1.6 Experiment 2
1.7 Model Exam (Unit 107)

2. APPLICATIONS OF MATRIX METHODS: FIXED POINT AND ABSORBING MARKOV CHAINS
2.1 Challenge Problem
2.2 Regular Transition Matrices
2.3 Fixed-Probability Vectors
2.4 Calculating a Fixed-Probability Vector
2.5 Experiment 1
2.6 A Fixed Probability Vector from a System of Linear Functions
2.7 Experiment 2
2.8 Experiment 3
2.9 Absorbing Markov Chains
2.10 A Second Challenge Problem
2.11 Standard Form for an Absorbing Markov Chain
2.12 Partitioning the Standard Form
2.13 Making Decision Based on Probability
2.14 The Probability of Reaching a Given Absorbing State
2.15 Experiment 4
2.16 Model Exam (Unit 111)

3. ANSWERS TO EXERCISES (UNIT 107)

4. ANSWERS TO MODEL EXAM (UNIT 107)

5. ANSWERS TO MODEL EXAM (UNIT 111)

APPENDIX A