Continuous-time dynamics

The probability distribution for a state variable X(t) is governed by the master equation

where the transition rate from state X to X' is denoted by w( X, X').

We make one key assumption: that the change in the state variable, i.e., X' — X, remains the same for different values of N. This assumption is cer­tainly met in birth-and-death processes, since jumps are restricted to be ±1 in binary choice models in Chapter 4, or +1 and —2 in the search models in

Chapter 9, from any state, regardless of the total number N. This assumption actually is concerned with the scaling properties or homogeneity properties of the transition rates. Loosely put, it means that each of the N microeconomic units may contribute approximately equally to the transition events. The transi­tion rates reflect combinatorial ways certain events take place in a short interval of time. To make this explicit, we express the transition rate as a function of the starting state X' and the jump (vector) r = X — X' and write and assume that

Using the same function, we can express the transition rate in the opposite direction as

A scaling property that is seemingly more general is

for some positive function f (N).

More generally, the transition rates may take the form of a power-series expansion in some inverse expression of N, such as N^—1:

where higher-order terms in N^—1 may represent higher-order interactions among microeconomic agents.

In terms of these transition rates, the master equation may be rewritten as

Example. A Markov chain with finite states. Suppose that a state variable X(t, ω) is defined on a probability space (Ω, F, P) with values in a finite state space IE = {1, 2,..., N}. We write

We assume that

uniformly in t as h ψ 0, where q_ki := — _k=_i q_i._k. We drop ω for brevity. In

this example we write the transition rate from state i to k, w(i, k), as q_i,_k.

Then the differential equation for p_ij is obtained by examining

where we note that Pr(X (t + h) = j | X (h) = i) = Pr(X (t) = j | X (0) = i) by the time homogeneity, which is assumed. By substituting the transition-rate expression into the above,

from which we obtain

The steady-state solution of this master equation, denoted by p _i, is obtained by setting the left-hand side of the master equation equal to zero:

^[1] This graph has vertices at states of the Markov chain, with edges connecting two states when the transition rates between the two states are nonzero.

^[1] See Aoki (1996a, p. 261) oor example.

that is,

3.2