From Martingales to Diffusion Generators

1Sigma-fields and information

A probability measure $\mathbb{P}$ on $(\mathrm{\Omega },\mathcal{F})$ is defined only on events $A\in \mathcal{F}$ and satisfies $$0\le \mathbb{P}(A)\le 1,\mathbb{P}(\mathrm{\Omega })=1,\mathbb{P}(\mathrm{\varnothing })=0.$$ If $A_1,A_2,\dots$ are pairwise disjoint events in $\mathcal{F}$, then countable additivity gives $$\mathbb{P}\left(\bigcup _{i=1}^{\mathrm{\infty }}{A}_i\right)=\sum _{i=1}^{\mathrm{\infty }}\mathbb{P}(A_i).$$

For two discrete random variables, $\sigma (X,Y)$ is generated by the common refinement of the partitions induced by $X$ and $Y$. More generally, for arbitrary random variables, $\sigma (X,Y)$ contains at least as much information as $\sigma (X)$: $$\sigma (X)\subseteq \sigma (X,Y).$$

2Conditional expectation

2.1From the discrete formula to a sigma-field viewpoint

For discrete random variables $X$ and $Y$, the familiar formula is $$g(y):=\mathbb{E}[X\mid Y=y]=\sum _{x}x\mathbb{P}(X=x\mid Y=y),$$ whenever $\mathbb{P}(Y=y)>0$. The law of total expectation becomes $$\mathbb{E}[X]=\mathbb{E}[\mathbb{E}[X\mid Y]]=\sum _y\mathbb{E}[X\mid Y=y]\mathbb{P}(Y=y).$$

The course mainly uses conditional expectation through its familiar discrete properties. The following general definition is included because it makes the role of filtrations especially transparent; its existence theorem belongs to the measure-theoretic background and is not developed here.

If $X\in L^2$, then $\mathbb{E}[X\mid \mathcal{G}]$ is the orthogonal projection of $X$ onto the closed subspace $L^2(\mathrm{\Omega },\mathcal{G},\mathbb{P})$ of $\mathcal{G}$-measurable square-integrable random variables.

2.2Recovery of the discrete formula

Assume $X$ and $Y$ are discrete and $\mathbb{E}|X|<\mathrm{\infty }$. For each $y$ with $\mathbb{P}(Y=y)>0$, define $$g(y)=\sum _{x}x\mathbb{P}(X=x\mid Y=y)=\frac{1}{\mathbb{P}(Y=y)}\sum _{x}x\mathbb{P}(X=x,Y=y),$$ and define $g(y)$ arbitrarily when $\mathbb{P}(Y=y)=0$. Set $Z=g(Y)$. Then $Z$ is $\sigma (Y)$-measurable. Every event in $\sigma (Y)$ is a union of level sets $\{Y=y\}$, so it suffices to verify the defining integral identity on each such level set. Indeed, $$\begin{array}{rl}{\int }_{\{Y=y\}}Z\mathrm{d}\mathbb{P}& =g(y)\mathbb{P}(Y=y)\\ & =\sum _{x}x\mathbb{P}(X=x,Y=y)\\ & ={\int }_{\{Y=y\}}X\mathrm{d}\mathbb{P}.\end{array}$$ Hence $g(Y)$ is a version of $\mathbb{E}[X\mid \sigma (Y)]$, usually abbreviated as $\mathbb{E}[X\mid Y]$.

Taking $A=\mathrm{\Omega }$ in the defining identity yields the general law of total expectation: $$\mathbb{E}[\mathbb{E}[X\mid \mathcal{G}]]=\mathbb{E}[X].$$

2.3Tower properties

This is often summarized as “the smaller sigma-field wins.”

For example, $\sigma (X_1,X_2,\dots )$ and $\sigma (Y_1,Y_2,\dots )$ are independent exactly when the random vectors $(X_1,X_2,\dots )$ and $(Y_1,Y_2,\dots )$ are independent.

2.4Linearity and conditional variance

For integrable random variables $X,Y$ and constants $a,b$, $$\mathbb{E}[aX+bY\mid \mathcal{G}]=a\mathbb{E}[X\mid \mathcal{G}]+b\mathbb{E}[Y\mid \mathcal{G}]\text{a.s.}$$ If $X\in L^2$, the conditional variance of $X$ given $\mathcal{G}$ is $$\mathrm{Var}(X\mid \mathcal{G}):=\mathbb{E}[(X-\mathbb{E}[X\mid \mathcal{G}]{)}^2|\mathcal{G}].$$ Expanding the square and using the pull-out property for $\mathcal{G}$-measurable factors gives $$\mathrm{Var}(X\mid \mathcal{G})=\mathbb{E}[X^2\mid \mathcal{G}]-(\mathbb{E}[X\mid \mathcal{G}]{)}^2.$$

3Stochastic processes, filtrations, and martingales

The natural filtration of a process $(X_t)$ is $${\mathcal{F}}_t^X:=\sigma (X_s:s\le t).$$ In discrete time, this reads ${\mathcal{F}}_k^X=\sigma (X_0,\dots ,X_k)$.

By total expectation, every martingale has constant mean: $$\mathbb{E}[X_{k+1}]=\mathbb{E}[\mathbb{E}[X_{k+1}\mid {\mathcal{F}}_k]]=\mathbb{E}[X_k].$$ The converse is false: constant mean is unconditional, whereas the martingale property must hold conditionally after every possible history.

3.1Examples

Let ${\mathcal{F}}_{\mathrm{\infty }}:=\sigma (\bigcup _{n\ge 0}{\mathcal{F}}_n)$. Lévy’s upward theorem completes the heuristic that $M_n$ is the best estimate of $X$ based on the information available by time $n$: $$M_n⟶\mathbb{E}[X\mid {\mathcal{F}}_{\mathrm{\infty }}]\text{almost surely and in }{L}^1.$$ In particular, if $X$ is ${\mathcal{F}}_{\mathrm{\infty }}$-measurable, then $M_n\to X$ almost surely and in $L^1$.

4Stopping times and optional stopping

In discrete time, $T$ takes values in ${\mathbb{N}}_0\cup \{\mathrm{\infty }\}$. Informally, by time $t$ one can determine whether stopping has occurred, without using future information.

Two common sufficient conditions are:

1. $T$ is bounded almost surely; or
2. $\mathbb{E}[T]<\mathrm{\infty }$ and the increments are uniformly bounded, meaning there is a constant $C<\mathrm{\infty }$ such that $|M_n-M_{n-1}|\le C$ almost surely for every $n\ge 1$.

In case (ii), $$|M_{n\wedge T}|\le |M_0|+CT,$$ and the integrable right-hand side implies uniform integrability.

The theorem formalizes the idea that an admissible stopping strategy cannot create positive expected profit from a fair game. The hypotheses matter: unbounded stopping rules without suitable integrability can invalidate the conclusion and may expose a player to arbitrarily large intermediate losses.

5Markov chains

5.1Markov property and transition probabilities

Let $(\mathrm{\Omega },\mathcal{F},\mathbb{P})$ be a probability space and let $(E,\mathcal{E})$ be a measurable state space, such as ${\mathbb{R}}^d$ with its Borel sigma-field. A stochastic process $(X_t{)}_{t\in T}$ taking values in $E$ consists of measurable maps $$X_t:(\mathrm{\Omega },\mathcal{F})⟶(E,\mathcal{E}).$$

Usually $({\mathcal{F}}_t)$ is the natural filtration. Informally, conditional on the present state, the future is independent of the past.

For a discrete-time process on a countable state space, the Markov property is equivalently $$\mathbb{P}(X_{n+1}=j\mid X_n=i,X_{n-1}=i_{n-1},\dots ,X_0=i_0)=\mathbb{P}(X_{n+1}=j\mid X_n=i)$$ whenever the conditioning event has positive probability.

If the one-step probability $\mathbb{P}(X_{n+1}=j\mid X_n=i)$ does not depend on $n$, the chain is time-homogeneous, and we write it as $p_{ij}$. Its $m$-step transition probability is $$p_{ij}^{(m)}:=\mathbb{P}(X_{n+m}=j\mid X_n=i),$$ which is independent of $n$ for a time-homogeneous chain. The Chapman–Kolmogorov equations state that $$p_{ij}^{(n+m)}=\sum _{k\in E}{p}_{ik}^{(n)}{p}_{kj}^{(m)}.$$ In matrix notation, $P^{(n+m)}=P^{(n)}{P}^{(m)}$.

5.2Harmonic and space–time harmonic functions

Subject to the required integrability, if $h$ is harmonic on all of $E$, then $(h(X_n))$ is a martingale with respect to the natural filtration.

Then $(h(X_n,n))$ is a martingale, provided it is integrable. For a time-inhomogeneous chain, $p_{xy}$ is replaced by the time-dependent kernel $p_n(x,y)$.

In continuous time, if $(P_t{)}_{t\ge 0}$ is the Markov semigroup, the analogous space–time relation is $$h(x,s)=P_{t}h(\cdot ,s+t)(x)={\int }_{E}h(y,s+t)P_t(x,\mathrm{d}y).$$ For standard Brownian motion, a sufficiently smooth space–time harmonic function satisfies the backward heat equation $$\frac{\partial h}{\partial s}+\frac{1}{2}\frac{{\partial }^{2}h}{\partial x^2}=0.$$

5.3Biased random walk and gambler’s ruin

Let ${\xi }_1,{\xi }_2,\dots$ be independent, let $p\in (0,1)$, and suppose $$\mathbb{P}({\xi }_k=1)=p,\mathbb{P}({\xi }_k=-1)=q:=1-p,$$ and define $S_n=s_0+\sum _{k=1}^n{\xi }_k$. Suppose first that $p\ne q$ and put $r=q/p$. The function $$g(x)=r^x$$ is harmonic because $$qg(x-1)+pg(x+1)=g(x).$$ Consequently, $(g(S_n))$ is a martingale.

Fix integers $a<s_0<b$ and let $$T:=inf\{n\ge 0:S_n\in \{a,b\}\}.$$ The walk exits the finite interval $\{a+1,\dots ,b-1\}$ almost surely; this is true in both the biased and symmetric cases. The stopped process $g(S_{n\wedge T})$ is bounded, so optional stopping gives $$r^{s_0}=\mathbb{E}[g(S_T)]=\mathbb{P}(S_T=a)r^a+\mathbb{P}(S_T=b)r^b.$$ Therefore, for $p\ne \frac{1}{2}$, $$\mathbb{P}(S_T=a)=\frac{r^{s_0}-r^b}{r^a-r^b},\mathbb{P}(S_T=b)=\frac{r^a-r^{s_0}}{r^a-r^b}.$$ For $p=q=\frac{1}{2}$, take the limit $r\to 1$ (or use that $S_n$ itself is a martingale) to obtain $$\mathbb{P}(S_T=a)=\frac{b-s_0}{b-a},\mathbb{P}(S_T=b)=\frac{s_0-a}{b-a}.$$

5.4Expected exit time for symmetric random walk

Now let $(S_n)$ be simple symmetric random walk with $S_0=0$, and let $a<0<b$ be integers. Although the expected time to hit a single nonzero level is infinite, the exit time $$T:=inf\{n\ge 0:S_n\in \{a,b\}\}$$ has finite expectation. To calculate it, define $$M_n:=S_n^2-n.$$ Since $$M_{n+1}-M_n=(S_n+{\xi }_{n+1}{)}^2-(n+1)-(S_n^2-n)=2S_n{\xi }_{n+1},$$ and $\mathbb{E}[{\xi }_{n+1}\mid {\mathcal{F}}_n]=0$, the process $(M_n)$ is a martingale.

For $k\ge 1$, the time $T\wedge k$ is bounded. Optional stopping gives $$0=\mathbb{E}[M_{T\wedge k}]=\mathbb{E}[S_{T\wedge k}^2]-\mathbb{E}[T\wedge k].$$ The stopped positions are bounded by $max\{|a|,|b|\}$, so dominated convergence applies to the first term; monotone convergence applies to the second. Hence $$\mathbb{E}[T]=\mathbb{E}[S_T^2].$$ Using the symmetric gambler’s-ruin probabilities, $$\mathbb{E}[T]=a^2\frac{b}{b-a}+b^2\frac{-a}{b-a}=-ab=|ab|.$$ In the special case $a=-b$, this becomes $\mathbb{E}[T]=a^2$, an instance of the parabolic scaling principle “time is of the order of distance squared.”

6Submartingales and maximal inequalities

For a submartingale, $\mathbb{E}[X_{n+1}]\ge \mathbb{E}[X_n]$. For example, if $Y_1,Y_2,\dots$ are independent and integrable with $\mathbb{E}[Y_i]=\mu \ge 0$, then $$X_n:=\sum _{i=1}^n{Y}_i,{\mathcal{F}}_n:=\sigma (Y_1,\dots ,Y_n)$$ defines a submartingale because $$\mathbb{E}[X_{n+1}\mid {\mathcal{F}}_n]=X_n+\mu \ge X_n.$$

The optional-sampling inequality says that if $(X_n)$ is a submartingale and $S\le T$ are bounded stopping times, then $$\mathbb{E}[X_S]\le \mathbb{E}[X_T].$$ The inequalities reverse for supermartingales. More general stopping times require additional conditions such as uniform integrability.

6.1Conditional Jensen inequality

If $\phi$ is convex and the relevant variables are integrable, conditional Jensen gives $$\phi (\mathbb{E}[X\mid \mathcal{G}])\le \mathbb{E}[\phi (X)\mid \mathcal{G}].$$ Consequently:

1. if $(X_n)$ is a martingale and $\phi$ is convex, then $(\phi (X_n))$ is a submartingale whenever it is integrable;
2. if $(X_n)$ is a submartingale and $\phi$ is convex and nondecreasing, then $(\phi (X_n))$ is a submartingale whenever it is integrable.

In particular, if $(X_n)$ is a martingale, then $(|X_n|)$ is a submartingale; if the martingale is square-integrable, then $(X_n^2)$ is also a submartingale.

7Martingale convergence

A particularly useful special case is $L^2$ boundedness.

8The invariance principle and Brownian motion

Let ${\xi }_1,{\xi }_2,\dots$ be i.i.d. with $\mathbb{E}[{\xi }_1]=0$ and $\mathrm{Var}({\xi }_1)=1$, and write $S_n=\sum _{k=1}^n{\xi }_k$. The central limit theorem states that $$\frac{S_n}{\sqrt{n}}\stackrel{d}{\to }N(0,1).$$ To retain the entire path rather than only its endpoint, define the rescaled step process $$Y_t^{(n)}:=\frac{S_{\lfloor nt\rfloor }}{\sqrt{n}},t\ge 0.$$ The floor is convenient because it preserves the discrete-time martingale as a piecewise-constant process. Linear interpolation is also valid for the functional limit theorem, although with the usual piecewise-constant filtration it uses the next increment between grid points and is not a martingale.

The word “invariance” means that, after centering and variance normalization, the same Brownian limit arises for any i.i.d. increment law with finite variance; the detailed distribution of ${\xi }_1$ does not affect the limit.

9Further properties of Brownian motion

Brownian motion is a Markov process and a martingale with respect to its natural filtration. It is also a Gaussian process: every finite vector $(B_{t_1},\dots ,B_{t_m})$ is multivariate normal. This follows because such a vector is a linear transformation of independent Gaussian increments.

9.1Quadratic variation

Let $${\pi }_n=\{0=t_{n,0}<t_{n,1}<\cdots <t_{n,m_n}=t\}$$ be deterministic partitions of $[0,t]$, with mesh $|{\pi }_n|:=\underset{k}{max}(t_{n,k}-t_{n,k-1})\to 0$. Then $$Q_n:=\sum _{k=1}^{m_n}(B_{t_{n,k}}-B_{t_{n,k-1}}{)}^2⟶t$$ in $L^2$ and hence in probability. Indeed, writing ${\mathrm{\Delta }}_{n,k}B=B_{t_{n,k}}-B_{t_{n,k-1}}$ and ${\mathrm{\Delta }}_{n,k}t=t_{n,k}-t_{n,k-1}$, $$\mathbb{E}[({\mathrm{\Delta }}_{n,k}B{)}^2]={\mathrm{\Delta }}_{n,k}t,\mathrm{Var}(({\mathrm{\Delta }}_{n,k}B{)}^2)=2({\mathrm{\Delta }}_{n,k}t{)}^2.$$ Independence of the increments gives $$\mathbb{E}[Q_n]=t,\mathrm{Var}(Q_n)=2\sum _k({\mathrm{\Delta }}_{n,k}t{)}^2\le 2t|{\pi }_n|⟶0.$$ For the dyadic partitions, the variances are summable, so the convergence is also almost sure by Chebyshev’s inequality and Borel–Cantelli. More generally, almost-sure convergence follows under suitable summability conditions on the meshes. This limit is denoted $$[B{]}_t=t$$ and is called the quadratic variation of Brownian motion.

9.2Transformations of Brownian motion

Several useful transformations follow from the defining properties.

1. Scaling. For any $a>0$, ${\left(\frac{1}{a}{B}_{a^{2}t}\right)}_{t\ge 0}$ is standard Brownian motion.
2. Time inversion. Define $X_0=0$ and $X_t=t{B}_{1/t}$ for $t>0$. Then $(X_t{)}_{t\ge 0}$ is standard Brownian motion.
3. Brownian bridge. On $0\le t\le 1$, define $Z_t=B_t-t{B}_1$. This centered Gaussian process has covariance $\mathrm{Cov}(Z_s,Z_t)=min(s,t)-st$ and the same law as Brownian motion conditioned to satisfy $B_1=0$. Moreover, $(Z_t{)}_{0\le t\le 1}$ is independent of $B_1$.

9.3Reflection principle and hitting times

For $a>0$ and $t>0$, the reflection principle gives $$\mathbb{P}(\underset{0\le s\le t}{max}{B}_s\ge a)=2\mathbb{P}(B_t\ge a)=\frac{2}{\sqrt{2\pi t}}{\int }_a^{\mathrm{\infty }}{e}^{-x^2/(2t)}\mathrm{d}x.$$ Let $$T_a:=inf\{t\ge 0:B_t=a\}.$$ Continuity gives $\{T_a\le t\}=\{\underset{0\le s\le t}{max}{B}_s\ge a\}$, so $$\mathbb{P}(T_a\le t)=2\mathbb{P}(B_t\ge a).$$ Differentiating yields the density $$f_{T_a}(t)=\frac{a}{\sqrt{2\pi }}{t}^{-3/2}\exp (-\frac{a^2}{2t}),t>0.$$ Consequently $\mathbb{P}(T_a<\mathrm{\infty })=1$, but $$\mathbb{E}[T_a]=\mathrm{\infty },$$ because $t{f}_{T_a}(t)$ is asymptotic to a positive constant times $t^{-1/2}$ as $t\to \mathrm{\infty }$. Brownian motion is recurrent in one dimension: almost surely it returns to every fixed level infinitely often.

9.4Brownian motion with drift and volatility

Given $\mu \in \mathbb{R}$ and $\sigma >0$, the process $$X_t=X_0+\mu t+\sigma B_t$$ is Brownian motion with drift $\mu$ and volatility $\sigma$. Its increments satisfy $$X_t-X_s\sim N(\mu (t-s),{\sigma }^2(t-s)).$$ The centered process $X_t-X_0-\mu t=\sigma B_t$ is a martingale, whereas $X_t$ itself is a martingale only when $\mu =0$.

10The Itô integral

Let $(B_t{)}_{0\le t\le T}$ be Brownian motion on a filtered probability space satisfying the usual conditions.

The use of the left endpoint is essential: $A_k$ may depend on information known by time $t_k$, but not on the future increment $B_{t_{k+1}}-B_{t_k}$.

The isometry extends the integral to every predictable process $H$ satisfying $$\mathbb{E}\left[{\int }_0^T{H}_t^2\mathrm{d}t\right]<\mathrm{\infty }.$$ Choose simple predictable $H^{(n)}$ with $$\mathbb{E}[{\int }_0^T|H_t^{(n)}-H_t{|}^2\mathrm{d}t]⟶0.$$ The isometry makes ${\int }_0^T{H}_t^{(n)}\mathrm{d}{B}_t$ Cauchy in $L^2$, and one defines $${\int }_0^T{H}_t\mathrm{d}{B}_t:=L^2\text{-}\underset{n\to \mathrm{\infty }}{lim}{\int }_0^T{H}_t^{(n)}\mathrm{d}{B}_t.$$ This definition is independent of the approximating sequence, preserves the Itô isometry, and produces a continuous square-integrable martingale $M_t={\int }_0^t{H}_s\mathrm{d}{B}_s$.

11Itô’s formula