![](\"\/..\/images\/markov_feb15_2018.png\")
<\/p>\nA Markov chain is a sequence of states through which a probability system can pass. It’s not so complex as it sounds. Consider the following example:<\/p>\n
Ms A must choose each day between a vegetarian lunch or a meat one. Here is a description of the parameters:<\/p>\n
1: veggie<\/p>\n
2: meat<\/p>\n
1,1: veggie today, then veggie tomorrow<\/p>\n
1,2: veggie today, but meat tomorrow<\/p>\n
2,1: meat today, veggie tomorrow<\/p>\n
2,2: meat today, meat tomorrow<\/p>\n
In the probability matrix, the entries are referred to by (row, column).<\/p>\n
Now let’s imagine the probabilities associated with each choice set are as follows:<\/p>\n
1,1: 0.55 (if veggie today, then 55% probability of veggie tomorrow as well)<\/p>\n
1,2: 0.45 (if veggie today, then 45% probability of meat tomorrow)<\/p>\n
2,1: 0.31 (if meat today, then 31% probability of veggie tomorrow)<\/p>\n
2,2: 0.69 (if meat today, then 69% probability of meat tomorrow, too)<\/p>\n
In matrix form:<\/p>\n
<\/p>\n
Such a matrix can be called a probability matrix<\/em>; another name is transition matrix<\/em>.<\/p>\n Observations:<\/p>\n 1) Being probabilities, each entry e must satisfy 0\u2264e\u22641 A square matrix that satsifies 1) and 2) is called stochastic<\/em>. <\/p>\n This is a good first step on our journey through Markov chains.<\/p>\n HTH:)<\/p>\n Sources:<\/u><\/p>\n Tan, Soo Tang. Applied Finite Mathematics<\/u>, 3rd Ed. Boston: PWS-Kent Publishing \u00a0\u00a0Company, 1990.<\/p>\n
\n2) The entries in a given row must sum to 1, since they represent all possibilites.
\n3) The diagonal entries (1,1 and 2,2) signify the probability of remaining the same.<\/p>\n