Detailed-balance conditions and stationary distributions
Agents also cluster into several groups, or change their minds and leave one cluster to join another in the model. We introduce the notion of stationary or equilibrium distributions, reversibility, and detailed-balance conditions, all of which are used extensively in this book.
We refer the reader to Kelly (1979, Chap. 1) as a convenient source of these notions. Some of the materials in Kelly go back to Kingman (1969), or to Kendall (1949).When there are K types of agents, we have earlier described one state vector as
where ni is a non negative integer that denotes the number of agents of type i, i = 1, 2,..., K. The vector n describes the clusterings of agents over K types. The total number of agents in the model is given by
In modeling closed binary models, n1 alone can serve as the state variable because n2 = N — n1.
The vector of fractions, n/N, is an empirical distribution. In the above and in what follows we omit the time variable from expressions such as ni (t).
As you notice in Kelly (1979), Markov chains are described in terms of conditional probabilities,
where the next time instant after t is denoted as t + 1 by a suitable choice of time unit, and where, without loss of generality, the state space is 5 = {1, 2,..., K}. The notion of the transition rate from state j to k is then defined as
for some small positive τ. We follow this practice of developing discrete-time and continuous-time versions in parallel when convenient or needed.
Before we discuss dynamics, which are determined once we specify transition rates between the states under conditions to rule out pathological behavior by Markov processes,[4] we describe how stationary or equilibrium distributions are determined by the transition rates.
Writing w (n, n') to denote the transition rate from state n to n', the equilibrium distribution satisfies
where π (n) is the equilibrium probability of state vector n, and where the summation is over all possible next states; for example, six possible next states in the case of an open binary model, and two in the case of a closed binary model. See the formulation of Kelly (1979, (1.3)). This equation states that the probability influx and outflux balance out in stationary states. The method for constructing a Markov process with these transition rates is standard (due to Feller). See Kelly (1983) for a short description.
In deriving the stationary or equilibrium distributions, we appeal to the notion of detailed-balance conditions extensively. The conditions are that there exists a set of positive numbers π (j), j ∈ 5, summing to unity, such that
for any state j, k ∈ S. These π(j) are the components of the equilibrium distribution. In the continuous-time version, the detailed-balance conditions are
for all j, k ∈ 5.
Kelly (1979, Sec. 1.2) shows that a stationary Markov chain is reversible if and only if the detailed-balance conditions hold. Similarly for a stationary Markov processes.
A necessary and sufficient condition for reversibility is the Kolmogorov cycle condition (Kendall 1959, Eq. (5.12)). Kingman shows that it is sufficient to consider two three-cycles (1969, Eqs. (17), (18)). See also Kelly (1979, Exercise 1.5.2).
We later show that some well-known distributions result from some specific choices for the transition rates.