Finite State Space, a Trial Run

* * *

In his famous article, Kolmogorov based his theory of Markov processes on what became known as Kolmogorov's forwardITL and backward equations. In an attempt to explain Kolmogorov's ideas, K. Itô took a crucial step when he suggested that Kolmogorov's forward equation can be thought of as describing the "flow of a vector field on the space of probability measures." The purpose of this chapter is to develop Itô's suggestion in the particularly easy case when the state space E is finite. The rest of the book is devoted to carrying out the analogous program when E = Rn.

where each of the maps θ1 [member of] (0, 1) [??] (0, θ1, 1 - θ1) [member of] I0, θ0 [member of] (0, 1) [??] (θ0, 0, 1 - θ0) [member of] I1 and θ0 [member of] (0, 1) [??] (θ0, 1-θ0, 0) [member of] I2 is a diffeomorphism. In particular, after a little thought it becomes clear that although, starting at each θ [member of] Θn, motion in Θn can be performed in (n - 1) independent directions, movement in some, or all, those directions may have restricted parity. For example, if θ [member of] I0, then movement from θ will require that the 0th coordinate be initially nondecreasing. Similarly, if θ = (1, 0, 0), at least initially, the 0th coordinate will have to be nonincreasing while both the 1st and 2nd coordinates will have to be nondecreasing.

For the reasons just discussed, it is best to describe the notion of a differentiable path in Θn in terms of one-sided paths. That is, we will say that t [member of] [0,∞)[??]θ(t) [member of] Θn is differentiable if the derivative of

[MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII] is a smooth curve,

in which case we will call [??](0) a tangent to Θn at θ(0). Further, given θ [member of] Θn, we will use Tθ(Θn) to denote the set of all vectors [??](0) which arise as the (right) derivative at t = 0 of a smooth t [member of] [0, ∞) [??] θ(t) [member of] Θn with θ(0) = θ. The following statement is essentially obvious.

Lemma 1.1.1. For each m [member of] Zn, let em [member of] Rn be the vector whose mth coordinate is 1 and whose other coordinates are 0. Then Θn is the convex hull of {e0, ..., en-1]. Moreover, if θ [member of] Θn, then b [member of] Rn is a tangent to Θn at θ if and only if [MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII] and [MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII]. In particular, Tθ(Θn) is a closed, convex cone in Rn.

As we anticipated, the shape of Tθ(Θn) undergoes radical changes as θ moves from [??]n to the faces making up [partial derivative]Θn In particular, the preceding description makes it difficult to spot any structure which plays here the role of model space the way Rn does in the study of n-dimensional differentiable manifolds. On the other hand, as yet, we have not taken advantage of the basic property enjoyed by Θn: it is a simplex. In particular, we should expect that the structure of Tθ(Θn) for all θ [member of] Θn can be understood in terms of the structure of spaces Tθ(Θn) as θ runs over the vertices e0, ..., en-1 of Θn. This expectation is given substance by the following simple result.

Lemma 1.1.2. For each 0 ≤ m ≤ n -1, a [MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII] if and only if

[MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII].

Moreover, for any θ [member of] Θn, b [member of] Tθ(Θn) if and only if there exist al [member of] [MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII],0 ≤ l ≤ n - 1 such that

(1.1.3) [MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII].

Proof: The first assertion is a special case of Lemma 1.1.1. In proving the second part, we use the notation [MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII], and [MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII].

If [MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII], where [MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII] for each l [member of] Zn, then bm = [MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII], and so

[MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII]

Hence, b [member of] Tθ(Θn)

To go the other way, we work by induction on n ≥ 2. Thus, suppose that θ [member of] Θ2 and b [member of] TθΘ2 are given. Clearly, either b = 0, and there is nothing to do, or b0 > 0 and 1- θ0 > 0, or b1 > 0 and 1 - θ1 > 0. Thus, we may and will assume that b1 > 0 and 1 - θ1 > 0, in which case we take a0 = 1/[1 - θ1] b and a1 = 0. Next, let n ≥ 3 and assume the result for n - 1. Reasoning as before, we reduce to the case when bn-1 > 0 and 1 - θn-1 > 0. Set

[MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII] and [MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII],

and observe that [??] [member of] Θn-1and [??] [member of] T[??]Θn-1. Using the induction hypothesis, find [MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII] so that [MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII]. Next, take al = 0 if l = n - 1 or θl = 0, and, when 0 [less than or equal to] l ≤ n - 2 and θl > 0, take

[MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII].

It is then an easy matter to check that [MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII] for each 0 [less than or equal to] l ≤ n - 1 and [MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII].

§1.1.2. Back to M1(Zn): We now transfer the above analysis back to M1(Zn). Namely, using µ [member of] M1(Zn) [??] θµ [member of] Θn to denote the inverse of the map θ [member of] Θn [??] µθ [member of] M1(Zn), we say that the curve t [member of] [0, ∞) [??] µt [member of] M1(Zn) is differentiable if [MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII] is differentiable. In order to describe the tangent corresponding to such a curve, first note that, since θµ = (µ({0}), ..., µ({n-1})), an equivalent formulation of the preceding is to say that that, for each φ [member of] C(Zn; R), the map t [member of] [0, ∞) [??] <φ, µt> [member of] R is differentiable. Thus, it is reasonable to identify the corresponding tangent [??]0 as the linear functional on C(Zn; R) given by

(1.1.4) [MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII].

Clearly, the relationship between [??]0 and [??](0) is

[MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII].

Hence Λµ is an element of the space Tµ(M1(Zn)) of tangent vectors to M1(Zn) at µ [member of] M1(Zn) if and only if (cf. Lemma 1.1.1)

(1.1.5) [MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII], for some [MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII].

Of course, this means that, like [MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII] is a closed, convex cone, this time of linear functionals on C(Zn; R). It also means (cf. Lemma 1.1.2) that each Λµ [member of] Tµ (M1(Zn)) admits the representation

(1.1.6) [MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII].

On the other hand, except when µ is a point mass, the representation in (1.1.6) is far from unique! In fact, as the proof of Lemma 1.1.2 makes abundantly clear, except at vertices there are innumerable ways of decomposing b [member of] TθΘn into al's, and each decomposition leads to a different representation in (1.1.6).

Remarks 1.1. 7. Our derivation of (1.1.6) is a straight forward computation which fails to reveal the reason why we should have expected its validity. To understand its intuitive origin, one needs to know the interpretation of convergence of measures in terms of coupling. For our purposes, suffice it to say that when v is close to µ, then there exists a map x [member of] Zn [??] P(x, ·) [member of] M1 Zn) such that v = ∫ P(x, ·) µ(dx) and P(x, ·) is close to δx for most (as measured by µ) x [member of] Zn. Hence, if t [??] µt is differentiable at t = 0, then it is reasonable to hope that µt = ∫ Pt(x, ·) µ(dx) where, for each x, t [??] Pt(x, ·) is differentiable at t = 0 and determines an element [MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII], which is, obviously, a candidate for [MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII]. In addition, this picture helps to explain the lack of uniqueness alluded to above. Namely, there are lots of couplings, and each one has the potential to give a different choice of [MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII] in (1.1.6). Unfortunately, a proof of (1.1.6) based on this line of reasoning eludes me. Nonetheless, as we will see in the next subsection, we can give a more intrinsic and revealing way to understand (1.1.6).

A second, and more mundane, remark deals with our notion of a tangent vector. Namely, we have implicitly adopted a particular coordinate system in which to describe tangent vectors. To be precise, our coordinates are the maps µ [??] <φ, µ> as φ runs through C(Zn; R), and we have described a tangent vector by saying how it acts, as a directional derivative, on these coordinate maps. Knowing this, the action of a tangent vector on a more general function is determined the chain rule. That is,

[MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII],

where [??](θ) [equivalent to] = F(µθ).

1.2 A More Intrinsic Approach

So far, all of our considerations have rested heavily on the relationship between M1(Zn) and Θn In order to prepare for the analogous considerations when the state space Zn is replaced by Rn it is important for us to develop another, more intrinsic, way of thinking about tangent vectors to M1(Zn).

Taking a hint from (1.1.6) and the first part of Remark 1.1.7, we begin by trying to develop a deeper understanding of the tangent spaces [MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII] to the point masses. For this purpose, we need to take care of a few preparations.