9 AR(p)序列的谱密度和Yule-Walker方程
9.1 AR(p)序列的谱密度
9.1.1 AR(p)序列的自协方差
因为AR(p)的平稳解为 X_t = A^{-1}(\mathscr B) \varepsilon_t = \sum_{j=0}^\infty \psi_j \varepsilon_{t-j} 由线性平稳列性质知\{X_t\}为零均值，自协方差函数为 \begin{align} \gamma_k = E(X_{t+k} X_t) = \sigma^2 \sum_{j=0}^\infty \psi_j \psi_{j+k}, \quad k=0, 1, \dots \tag{9.1} \end{align}
设1<\rho<\min\{|z_j|\}，则\psi_j = o(\rho^{-j}), 有 \begin{align} |\gamma_k| \leq & \sigma^2 \sum_{j=0}^\infty |\psi_j| \cdot |\psi_{j+k}| \leq \sigma^2 (\sum_{j=0}^\infty |\psi_j|) (\sum_{l=k}^\infty |\psi_l|) \\ \leq & c_0 \sum_{l=k}^\infty \rho^{-l} \leq c_1 \rho^{-k} \tag{9.2} \end{align} 即\{\gamma_k\}负指数衰减。
\{X_t\}序列前后的相关减小很快， 称为时间序列的短记忆性。 征根离单位圆越远\{\gamma_k\}衰减越快。
9.1.2 AR(p)的谱密度
由线性平稳列的谱密度公式得平稳解的谱密度 \begin{aligned} f(\lambda) = \frac{\sigma^2}{2\pi} \left| \sum_{j=0}^\infty \psi_j e^{ij\lambda} \right|^2 \end{aligned} 而\sum \psi_j z^j = 1/A(z)所以 \begin{align} f(\lambda) = \frac{\sigma^2}{2\pi} \frac{1}{\left| A(e^{i\lambda}) \right|^2} \tag{9.3} \end{align} f(\lambda)是一个恒正的偶函数。 如果A(z)有靠近单位圆的根\rho_j e^{i\lambda_j}则 |A(e^{i\lambda_j})|会接近零， 造成谱密度在\lambda=\lambda_j处有一个峰值。
9.1.3 谱密度的自协方差函数反演公式
谱密度的定义是满足 \begin{aligned} \gamma_k = \int_{-\pi}^\pi e^{ik\lambda} f(\lambda) d\lambda \end{aligned} 的非负可积函数。上式是一个Fourier级数系数的公式(差一个常数)。
在\{\gamma_k\}满足一定条件下f(\lambda)必存在， 且可表成\{\gamma_k\}的Fourier级数。
定理9.1 如果平稳序列\{X_t\}的自协方差函数\{\gamma_k\} 绝对可和: \sum|\gamma_k|<\infty, 则\{X_t\}有谱密度 \begin{align} f(\lambda)=\frac{1}{2\pi} \sum_{k=-\infty}^\infty \gamma_k e^{-ik\lambda}. \tag{9.4} \end{align}
由于谱密度是实值函数, 所以(9.4)还可以写成 f(\lambda)=\frac{1}{2\pi}\sum_{k=-\infty}^\infty \gamma_k \cos(k\lambda) = \frac{1}{2\pi} \left[ \gamma_0 + 2\sum_{k=1}^\infty \gamma_k \cos(k\lambda)\right].
证明: 因为\{\gamma_k\}绝对可和所以(9.4)右边绝对一致收敛, f(\lambda)连续。 于是积分与级数可交换： \begin{aligned} \int_{-\pi}^\pi f(\lambda) e^{ij\lambda}\ d\lambda =\frac{1}{2\pi}\sum_{k=-\infty}^\infty \gamma_k \int_{-\pi}^\pi e^{-i(k-j)\lambda} \ d\lambda =\gamma_j. \end{aligned}
还要验证f(\lambda)非负。 若X_1, \dots, X_N为\{X_t\}的观测值， \begin{aligned} I_N(\lambda) = \frac{1}{2\pi N} \left| \sum_{j=1}^N X_j e^{ij\lambda} \right|^2, \quad \lambda \in [-\pi,\pi] \end{aligned} 称为X_1,\ldots, X_N的周期图。 令f_N(\lambda) = E I_N(\lambda), 则f_N(\lambda) \geq 0，于是 \begin{aligned} 0 \leq& f_N(\lambda) = \frac{1}{2\pi N} \sum_{k=1}^N \sum_{j=1}^N \gamma_{k-j} e^{-i(k-j)\lambda} \\ =& \frac{1}{2\pi N} \sum_{m=1-N}^{N-1} (N-|m|)\gamma_{m} e^{-im\lambda} \\ =& \frac{1}{2\pi} \sum_{m=1-N}^{N-1}\gamma_{m} e^{-im\lambda} -\frac{1}{2\pi N} \sum_{m=1-N}^{N-1}|m|\gamma_{m} e^{-im\lambda}. \end{aligned} \tag{*} 由Kronecker引理知后一项趋于0, 于是 \begin{aligned} f(\lambda) = \lim_{N\to \infty} f_N(\lambda) \geq 0. \end{aligned}
○○○○○○
附注：(*)式的二重求和的简化
\begin{aligned} & \frac{1}{2\pi N} \sum_{k=1}^N \sum_{j=1}^N \gamma_{k-j} e^{-i(k-j)\lambda} \\ =& \frac{1}{2\pi N} \sum_{k=1}^N \sum_{m=k-N}^{k-1} \gamma_{m} e^{-im\lambda} \quad (\text{令}m = k-j) \end{aligned} 交换m与k的求和次序。因为关于m的条件为 \begin{aligned} k - N \leq m \leq k - 1 \end{aligned} 所以k \leq m+N, k \geq m+1， 求和变为 \begin{aligned} \frac{1}{2\pi N} \sum_{m=1-N}^{N-1} \sum_{k=\max(m+1,1)}^{k=\min(m+N,N)} \gamma_{m} e^{-im\lambda} \end{aligned} 因为m \geq 0时k的求和从m+1到N有N-m项， m<0时k的求和从1到m+N=N-|m|有N-|m|项，所以求和变为 \begin{aligned} \frac{1}{2\pi N} \sum_{m=1-N}^{N-1} (N - |m|) \gamma_{m} e^{-im\lambda} \end{aligned}
附注：Kronecker引理： 设复数或实数级数\sum_{j=1}^n x_j收敛， 非负实数列{b_n}单调不减趋于+\infty， 则 \lim_{n\to\infty} \frac{1}{b_n} \sum_{j=1}^n b_j x_j = 0 .
见B.2。
○○○○○○
推论9.1 AR(p)的平稳解序列\{X_t\}有谱密度 \begin{aligned} f(\lambda)=\frac{1}{2\pi}\sum_{k=-\infty}^\infty \gamma_k e^{-ik\lambda} =\frac{\sigma^2}{2\pi}\frac{1}{|A(e^{i\lambda})|^2}. \end{aligned}
9.2 Yule-Walker方程
9.2.1 白噪声列与平稳解的关系
A(\mathscr B) X_t = \varepsilon_t的平稳解为 \begin{aligned} X_t = A^{-1}(\mathscr B) \varepsilon_t = \sum_{j=0}^\infty \psi_j \varepsilon_{t-j} \end{aligned} 对k \geq 1由控制收敛定理或内积的连续性得 \begin{aligned} E(X_t \varepsilon_{t+k} ) = \sum_{j=0}^\infty \psi_j E(\varepsilon_{t-j} \varepsilon_{t+k}) = 0 \end{aligned} 即X_t与未来的输入不相关。
如果\{\varepsilon_t\}是独立白噪声则X_t与未来的输入独立。
9.2.2 AR序列的等价定义
设a_1, \dots, a_p为实数， \{ \varepsilon_t \}为WN(0, \sigma^2), 若平稳列\{ X_t, t \in \mathbb Z \}满足 X_t = a_1 X_{t-1} + \dots + a_p x_{t-p} + \varepsilon_t, \ t \in \mathbb Z, 且\varepsilon_{t+k}, k=1,2,\dots与\{ X_s: s \leq t \}互不相关， 则称\{ X_t \}为AR(p)序列。
可以看出，两个定义是等价的。 由前面的推导可知原始定义可以导出等价定义中的性质。 反之，如果等价定义成立， A(z)=1 - a_1 z - \dots - a_p z^p一定满足最小相位性。
9.2.3 Yule-Walker方程
在AR(p)模型 X_t = a_1 X_{t-1} + \dots + a_p X_{t-p} + \varepsilon_{t-p} 两边同时乘以X_{t-k}(k \geq 1)后取期望， 有 E(X_t X_{t-k}) = a_1 E(X_{t-1} X_{t-k}) + \dots + a_p E(X_{t-p} X_{t-k}) + E(\varepsilon_t X_{t-k}) 即有 \gamma_k = a_1 \gamma_{k-1} + \dots + a_p \gamma_{k-p} + 0
所以，对k \geq 1，有递推式 \gamma_k = a_1 \gamma_{k-1} + \dots + a_p \gamma_{k-p} .
k \geq 1时，\gamma_k满足齐次线性差分方程 A(\mathscr B) \gamma_k = 0 .
将k=1,2,\dots,p的方程写成 \begin{aligned} \gamma_1 =& a_1 \gamma_0 + a_2 \gamma_1 + \dots + a_p \gamma_{p-1} \\ \gamma_2 =& a_1 \gamma_1 + a_2 \gamma_0 + \dots + a_p \gamma_{p-2} \\ \vdots =& \vdots \\ \gamma_p =& a_1 \gamma_{p-1} + a_2 \gamma_{p-2} + \dots + a_p \gamma_0 \end{aligned} 写成矩阵形式： \begin{aligned} \left(\begin{matrix} \gamma_1 \\ \gamma_2 \\ \gamma_3 \\ \vdots \\ \gamma_p \end{matrix}\right) =& \left(\begin{matrix} \gamma_0 & \gamma_1 & \cdots & \gamma_{p-1} \\ \gamma_1 & \gamma_0 & \cdots & \gamma_{p-2} \\ \gamma_2 & \gamma_1 & \cdots & \gamma_{p-3} \\ \vdots & \ddots & \ddots & \vdots \\ \gamma_{p-1} & \gamma_{p-2} & \cdots & \gamma_0 \end{matrix}\right) \left(\begin{matrix} a_1 \\ a_2 \\ \vdots \\ a_p \end{matrix}\right) \end{aligned} 记 \begin{aligned} \boldsymbol{\gamma}_p =& \left(\begin{matrix} \gamma_1 \\ \gamma_2 \\ \gamma_3 \\ \vdots \\ \gamma_p \end{matrix}\right), \ \Gamma_p = \left(\begin{matrix} \gamma_0 & \gamma_1 & \cdots & \gamma_{p-1} \\ \gamma_1 & \gamma_0 & \cdots & \gamma_{p-2} \\ \gamma_2 & \gamma_1 & \cdots & \gamma_{p-3} \\ \vdots & \ddots & \ddots & \vdots \\ \gamma_{p-1} & \gamma_{p-2} & \cdots & \gamma_0 \end{matrix}\right), \ \boldsymbol{a}_p = \left(\begin{matrix} a_1 \\ a_2 \\ \vdots \\ a_p \end{matrix}\right) \end{aligned} 有 \begin{aligned} & \Gamma_p \boldsymbol{a}_p = \boldsymbol{\gamma}_p \end{aligned}
对\gamma_0，
\begin{aligned}
E[X_t \varepsilon_t]
=& E[ (a_1 X_{t-1} + \dots + a_p X_{t-p} + \varepsilon_t) \varepsilon_t ] \\
=& 0 + \sigma^2 = \sigma^2 .
\end{aligned}
在X_t = a_1 X_{t-1} + \dots + a_p X_{t-p} + \varepsilon_t两边同时乘以X_t后取期望，得
\begin{aligned}
\gamma_0 =& a_1 \gamma_1 + \dots + a_p \gamma_p + E[X_t \varepsilon_t] \\
=& a_1 \gamma_1 + \dots + a_p \gamma_p + \sigma^2 .
\end{aligned}
上式另一证明为： \begin{aligned} \gamma_0 =& E X_t^2 = E \left( \sum_{j=1}^p a_j X_{t-j} + \varepsilon_t \right)^2 \\ =& E \left(\sum_{j=1}^p a_j X_{t-j}\right)^2 + E \varepsilon_t^2 \\ =& \boldsymbol{a}_p^T \Gamma_p \boldsymbol{a}_p + \sigma^2 \\ =& \boldsymbol{a}_p^T \boldsymbol{\gamma}_p + \sigma^2 \\ =& a_1 \gamma_1 + a_2\gamma_2 + \dots + a_p \gamma_p + \sigma^2 \end{aligned}
于是 \begin{aligned} \gamma_k =& a_1 \gamma_{k-1} + a_2\gamma_{k-2} + \dots + a_p \gamma_{k-p}, \ k \geq 1 \\ \gamma_0 =& a_1 \gamma_1 + a_2\gamma_2 + \dots + a_p \gamma_p + \sigma^2 . \end{aligned} 用推移算子写成 \begin{aligned} A(\mathscr B) \gamma_k =& 0, \ k \geq 1 \\ A(\mathscr B) \gamma_0 =& \sigma^2 . \end{aligned}
对n \geq p，令 \begin{aligned} \boldsymbol{\gamma}_n =& \left(\begin{matrix} \gamma_1 \\ \gamma_2 \\ \gamma_3 \\ \vdots \\ \gamma_n \end{matrix}\right), \ \boldsymbol{a}_n = \left(\begin{matrix} a_1 \\ a_2 \\ \vdots \\ a_p \\ 0 \\ \vdots \\ 0 \end{matrix}\right) \end{aligned} 仍有 \begin{align} \Gamma_n \boldsymbol{a}_n = \boldsymbol{\gamma}_n \tag{9.5} \end{align} 在节22.1将可以看到， 对于一般的平稳列\{X_t\}， \boldsymbol{a}_n是用X_{t-1}, \dots, X_{t-n}预测X_t时的最优线性预测系数。
为了证明(9.5)， 对n \geq p, 把X_t, X_{t+1}, \dots, X_{t+n-1}的递推式写成矩阵形式得 \begin{align} & \left( \begin{array}{l} X_t\\ X_{t+1}\\ \vdots \\ X_{t+n-1} \end{array} \right) \\ = & \left( \begin{array}{llll} X_{t-1} & X_{t-2} & \dots &X_{t-n}\\ X_{t} & X_{t-1} & \dots &X_{t+1-n}\\ \vdots & \vdots & & \vdots \\ X_{t+n-2} & X_{t+n-3} & \dots & X_{t-1}\\ \end{array} \right) \boldsymbol{a}_n + \left( \begin{array}{l} \varepsilon_t\\ \varepsilon_{t+1}\\ \vdots \\ \varepsilon_{t+n-1} \end{array} \right), \tag{9.6} \end{align} 在(9.6)两边同时乘上X_{t-1}后取数学期望, 利用X_t与未来输入的不相关性就可以证明(9.5)式。
定理9.2 (Yule-Walker方程) AR(p)序列的自协方差函数满足 \begin{align} \boldsymbol{\gamma}_n = \Gamma_n\boldsymbol{a}_n, \quad \gamma_0 = \boldsymbol{\gamma}_n^T \boldsymbol{a}_n + \sigma^2, \quad n\geq p, \tag{9.7} \end{align} 即 \begin{align} \gamma_k =& a_1 \gamma_{k-1} + a_2 \gamma_{k-2} + \dots + a_p \gamma_{k-p}, \quad k\geq 1 \tag{9.8} \\ A(\mathscr B) \gamma_k =& 0, \quad k \geq 1 \\ A(\mathscr B) \gamma_0 =& \gamma_0 - a_1 \gamma_1 - a_2 \gamma_2 - \dots - a_p \gamma_p = \sigma^2 \tag{9.9} \end{align} 特别地，当n=p时 \begin{align} & \Gamma_p \left(\begin{array}{c} a_1 \\ a_2 \\ \dots \\ a_p \end{array} \right) = \left(\begin{array}{c} \gamma_1 \\ \gamma_2 \\ \dots \\ \gamma_p \end{array} \right) \tag{9.10} \end{align}
记\phi_0=1, \phi_1 = -a_1, \dots, \phi_p=-a_p, 则A(z) = \sum_{j=0}^p \phi_j z^j， AR模型可写成\sum_{j=0}^p \phi_j X_{t-j} = \varepsilon_t。
Yule-Walker方程可写成 \begin{aligned} \left(\begin{array}{cccc} \gamma_0 & \gamma_1 & \cdots & \gamma_p \\ \gamma_1 & \gamma_0 & \cdots & \gamma_{p-1} \\ \vdots & \vdots & & \vdots \\ \gamma_p & \gamma_{p-1} & \cdots & \gamma_0 \end{array}\right) \left(\begin{array}{c} \phi_0 \\ \phi_1 \\ \vdots \\ \phi_p \end{array}\right) = \left(\begin{array}{c} \sigma^2 \\ 0 \\ \vdots \\ 0 \end{array}\right) \end{aligned}
9.3 自协方差函数的周期性
对k<0定义\psi_k=0。
推论9.2 AR(p)序列的自协方差函数\{\gamma_k\}满足和 AR(p)模型A(\mathscr B) X_t = \varepsilon_t相应的差分方程 \begin{aligned} \gamma_k - (a_1 \gamma_{k-1} + a_2 \gamma_{k-2} + \dots + a_p \gamma_{k-p}) = \sigma^2 \psi_{-k}, \quad k \in \mathbb Z . \end{aligned}
证明: k \geq 0时即定理结论。对k<0， \begin{aligned} & \gamma_k - (a_1 \gamma_{k-1} + a_2 \gamma_{k-2} + \dots + a_p \gamma_{k-p}) \\ =& E\left[ X_{t-k} ( X_t - \sum_{j=1}^p a_j X_{t-j} ) \right] \\ =& E(X_{t-k} \varepsilon_t) = E \left[ \sum_{j=0}^\infty \psi_j \varepsilon_{t-k-j} \varepsilon_t \right] = \sigma^2 \psi_{-k} . \end{aligned}
○○○○○○
设A(z)有p个互异根z_j=\rho_j e^{i\lambda_j}, j=1,\dots,p， 可以证明（略） \begin{align} \gamma_t =& A^{-1}(\mathscr B) \sigma^2 \psi_{-t} \\ =& \sigma^2 \sum_{j=1}^p c_j A^{-1}(z_j^{-1}) z_j^{-t} \\ =& \sigma^2 \sum_{j=1}^p A_j \rho_j^{-t} \cos(\lambda_j t + \theta_j), \quad t \geq 0 . \tag{9.11} \end{align} 可见如果\{z_j\}中有靠近单位圆的复根则\{\gamma_k\} 的衰减振荡特性会显现出来。
例9.1 AR(4)模型1: \begin{aligned} z_1,z_2 = 1.09e^{\pm i\pi/3}, \quad z_3,z_4 = 1.098e^{\pm i 2\pi/3} \end{aligned} 周期为2\pi/(\pi/3)=6和2\pi/(2\pi/3)=3。
AR(4)模型2: \begin{aligned} z_1,z_2 = 1.264e^{\pm i\pi/3}, \quad z_3,z_4 = 1.273e^{\pm i 2\pi/3} \end{aligned}
AR(4)模型3: \begin{aligned} z_1,z_2 = 1.635e^{\pm i\pi/3}, \quad z_3,z_4 = 1.647e^{\pm i 2\pi/3} \end{aligned}
程序演示:
library(polynom)
demo.ar.roots <- function(){
n <- 1024
rtlis <- list(c(complex(mod=1.09, arg=pi/3*c(1,-1)),
complex(mod=1.098, arg=pi*2/3*c(1,-1))),
c(complex(mod=1.264, arg=pi/3*c(1,-1)),
complex(mod=1.273, arg=pi*2/3*c(1,-1))),
c(complex(mod=1.635, arg=pi/3*c(1,-1)),
complex(mod=1.647, arg=pi*2/3*c(1,-1))),
complex(mod=1.02, arg=pi/6*c(1,-1)),
complex(mod=1.02, arg=pi/2*c(1,-1)),
c(complex(mod=1.05, arg=pi/6*c(1,-1)),
complex(mod=1.05, arg=pi/2*c(1,-1))))
tits <- c("AR(4): 1.09exp(+-i pi/3), 1.098exp(+-i 2pi/3)",
"AR(4): 1.264exp(+-i pi/3), 1.273exp(+-i 2pi/3)",
"AR(4): 1.635exp(+-i pi/3), 1.647exp(+-i 2pi/3)",
"AR(2): mod=1.02 arg=+-pi/6",
"AR(2): mod=1.02 arg=+-pi/2",
"AR(4): mod=1.05 arg=+-pi/6,+-pi/2")
tits <- c(
expression(paste("AR(4):",
list(1.09*e^{phantom(.) %+-% i*frac(pi,3)},
1.098*e^{phantom(.) %+-% i*frac(2*pi,3)}) )),
expression(paste("AR(4):",
list(1.264*e^{phantom(.) %+-% i*frac(pi,3)},
1.273*e^{phantom(.) %+-% i*frac(2*pi,3)}) )),
expression(paste("AR(4):",
list(1.635*e^{phantom(.) %+-% i*frac(pi,3)},
1.647*e^{phantom(.) %+-% i*frac(2*pi,3)}) )),
expression(paste("AR(2):",
list(1.02*e^{phantom(.) %+-% i*frac(pi,6)}) )),
expression(paste("AR(2):",
list(1.02*e^{phantom(.) %+-% i*frac(pi,2)}) )),
expression(paste("AR(4):",
list(1.05*e^{phantom(.) %+-% i*frac(pi,6)},
1.05*e^{phantom(.) %+-% i*frac(pi,2)}) ))
)
oldpar <- par(mfrow=c(3,1), mar=c(2,2,0,0),
mgp=c(2, 0.5, 0), oma=c(0,0,2,0))
on.exit(par(oldpar))
for(ii in seq(along=rtlis)){
rt = rtlis[[ii]]
y <- ar.gen(n, rt, sigma=1.0, by.roots=TRUE,
plot.it=FALSE)
plot(window(y, 1, 60))
acf(y)
##spectrum(y, taper=0.2)
ar.true.spectrum(attr(y, "a"), title="")
mtext(tits[ii], side=3, outer=TRUE)
}
}
demo.ar.roots()## Warning in polynomial(p): 强制改变时丢弃了虚数部分
## Warning in polynomial(p): 强制改变时丢弃了虚数部分
## Warning in polynomial(p): 强制改变时丢弃了虚数部分
9.4 自协方差函数的正定性
AR(p)平稳解唯一故自协方差函数可被自回归系数和白噪声方差唯一决定。 反之， 若\Gamma_p正定则根据Yule-Walker方程可以从 \gamma_0, \gamma_1, \dots, \gamma_p解出a_1, \dots, a_p, \sigma^2: \begin{align} \boldsymbol{a}_p = \Gamma_p^{-1} \boldsymbol{\gamma}_p, \quad \sigma^2 = \gamma_0 - \boldsymbol{\gamma}_p^T \Gamma_p^{-1} \boldsymbol{\gamma}_p. \tag{9.12} \end{align}
定理9.3 设\Gamma_n是实值平稳序列\{X_t\}的n阶自协方差矩阵, \gamma_0>0。
(1) 如果\{X_t\}的谱密度f(\lambda)存在, 则对n\geq 1，\Gamma_n正定；
(2) 如果\lim_{k \to \infty} \gamma_k = 0， 则对n \geq 1，\Gamma_n正定。
引理9.1 对实平稳列\{X_t\}，设其自协方差阵为\Gamma_n, n \in \mathbb N； 设其谱函数为F(\lambda)。 \forall \boldsymbol{b}=(b_1, \dots, b_n) \in \mathbb C^n有 \begin{aligned} \boldsymbol{b}^* \Gamma_n \boldsymbol{b} = \int_{-\pi}^\pi \left| \sum_{j=1}^n b_j e^{ij\lambda} \right|^2 dF(\lambda) . \end{aligned} 若\{X_t\}有谱密度f(\lambda)则 \begin{aligned} \boldsymbol{b}^* \Gamma_n \boldsymbol{b} = \int_{-\pi}^\pi \left| \sum_{j=1}^n b_j e^{ij\lambda} \right|^2 f(\lambda) d\lambda . \end{aligned}
引理证明：
\begin{aligned} & \boldsymbol{b}^* \Gamma_n \boldsymbol{b} = \sum_{j=1}^n \sum_{k=1}^n \bar b_j b_k \gamma_{k-j} \\ =& \sum_{j=1}^n \sum_{k=1}^n \bar b_j b_k \int_{-\pi}^\pi e^{i(k-j)\lambda} dF(\lambda) \\ =& \int_{-\pi}^\pi \sum_{j=1}^n \sum_{k=1}^n \bar b_j b_k e^{-ij\lambda} e^{ik\lambda} dF(\lambda) \\ =& \int_{-\pi}^\pi \overline{\left( \sum_{j=1}^n b_j e^{ij\lambda} \right)} \left( \sum_{k=1}^n b_k e^{ik\lambda} \right) \,dF(\lambda)\\ =& \int_{-\pi}^\pi \left| \sum_{j=1}^n b_j e^{ij\lambda} \right|^2 dF(\lambda) . \end{aligned}
定理证明：
(1) 对\boldsymbol{b} = (b_1, \dots, b_n)^T， \sum_{j=1}^n b_j z^{j-1} 至多有n-1个零点。\gamma_0 = \int_{-\pi}^\pi f(\lambda) d\lambda > 0, 于是 \begin{aligned} \boldsymbol{b}^T \Gamma_n \boldsymbol{b} = \int_{-\pi}^\pi \left| \sum_{j=1}^n b_j e^{ij\lambda} \right|^2 f(\lambda) d \lambda > 0 . \end{aligned}
(2) 用反证法。
设\Gamma_n正定,
\det(\Gamma_{n+1})=0和EX_t=0
(非零均值情况只要减去均值).
定义
\boldsymbol{X}_n=(X_1,X_{2},\dots, X_n)^T
对任何实向量
\boldsymbol{b}=(b_1,b_2,\dots,b_n)^T \neq 0 有
E(\boldsymbol{b}^T \boldsymbol{X}_n)^2
=\boldsymbol{b}^T \Gamma_n \boldsymbol{b} >0,
且由|\Gamma_{n+1}|=0知存在
\boldsymbol{a} =(a_1,a_2,\dots,a_{n+1})^T \neq 0, a_{n+1}\neq 0使得
E(\boldsymbol{a}^T \boldsymbol{X}_{n+1})^2
= \boldsymbol{a} ^T \Gamma_{n+1} \boldsymbol{a}
=0.
于是
\boldsymbol{a}^T \boldsymbol{X}_{n+1} = a_1 X_1 + a_2 X_2 + \dots + a_{n+1} X_{n+1}=0
a.s.成立, X_{n+1}可以由\boldsymbol{X}_n线性表示:
\begin{aligned}
X_{n+1} = -\frac{a_n}{a_{n+1}} X_{n}
- \frac{a_{n-1}}{a_{n+1}} X_{n-1} - \dots
- \frac{a_1}{a_{n+1}} X_{1}, \quad
\text{a.s.},
\end{aligned}
利用\{X_t\}的平稳性知道 \begin{aligned} X_t = -\frac{a_n}{a_{n+1}} X_{t-1} - \frac{a_{n-1}}{a_{n+1}} X_{t-2} - \dots - \frac{a_1}{a_{n+1}} X_{t-n}, \quad \text{a.s.}, \quad t \in \mathbb Z . \end{aligned}
递推知对任何k\geq 1, X_{n+k} 可以由X_1,X_2,\dots,X_n线性表示, 即有实向量 \boldsymbol{\alpha} \stackrel{\triangle}{=} \boldsymbol{\alpha}^{(k)} \stackrel{\triangle}{=} (\alpha_1^{(k)}, \dots, \alpha_n^{(k)})^T 使得 X_{n+k}=(\boldsymbol{\alpha}^{(k)})^T \boldsymbol{X}_n.
X_{n+k}被\boldsymbol{X}_n线性表示， 说明X_{n+k}与X_1,\dots,X_n有强的相关， 而定理假设是\gamma_k \to 0， 又说明X_{n+k}与X_1,\dots,X_n的相关性要趋于零， 这就会有矛盾，下面把矛盾严格表述。
用 0 < \lambda_1 \leq \lambda_2 \dots \leq \lambda_n
表示\Gamma_n的特征值,
则有正交矩阵 B使得
B \Gamma_n B^T = \Lambda
= \text{diag}(\lambda_1, \lambda_2,\dots,\lambda_n).
用|\boldsymbol{\alpha}^{(k)}|
表示\boldsymbol{\alpha}^{(k)}的欧氏模, 则有
\begin{aligned}
\gamma_0 =& E X_{n+k}^2
= E((\boldsymbol{\alpha}^{(k)})^T \boldsymbol{X}_n)^2
= (\boldsymbol{\alpha}^{(k)})^T \Gamma_n \boldsymbol{\alpha}^{(k)} \\
=& ( (\boldsymbol{\alpha}^{(k)})^T B^T)
(B \Gamma_n B^T)
(B \boldsymbol{\alpha}^{(k)} )\\
=& ( (\boldsymbol{\alpha}^{(k)})^T B^T)
\Lambda
(B \boldsymbol{\alpha}^{(k)} )\\
\geq& \lambda_1 (B \boldsymbol{\alpha}^{(k)})^T
(B \boldsymbol{\alpha}^{(k)} )
= \lambda_1 |\boldsymbol{\alpha}^{(k)}|^2.
\end{aligned}
即有|\boldsymbol{\alpha}^{(k)} | \leq \sqrt{\gamma_0/\lambda_1} < \infty.
另一方面 \begin{aligned} \gamma_0 =& E( (\boldsymbol{\alpha}^{(k)})^T \boldsymbol{X}_n \cdot X_{n+k}) = (\boldsymbol{\alpha}^{(k)})^T E(\boldsymbol{X}_n X_{n+k} ) \\ =& (\boldsymbol{\alpha}^{(k)})^T (\gamma_{n+k-1}, \gamma_{n+k-2}, \dots,\gamma_{k})^T\\ \leq & |\boldsymbol{\alpha}^{(k)}| \left(\sum_{j=0}^{n-1}\gamma_{j+k}^2\right)^{1/2}\\ \leq & (\gamma_0/\lambda_1)^{1/2} \left(\sum_{j=0}^{n-1}\gamma_{k+j}^2\right)^{1/2}\\ \to & 0. \quad (\text{当}k\to \infty) \end{aligned} 这与\gamma_0>0矛盾, 故 \det(\Gamma_{n+1})=0不成立.
○○○○○○
推论9.3 系数平方可和的线性平稳序列的自协方差阵总是正定的。
这是上面的定理与定理6.2的推论。
对平稳列\{X_n \}的自协方差函数\{\gamma_j, j \in \mathbb Z\}， 如果\{X_n \}自协方差阵总是正定的， 则称自协方差函数\{\gamma_j, j \in \mathbb Z\}是正定序列。
定理9.4 设实平稳列\{ X_t \}的谱函数F(\lambda)是阶梯函数。 如果F(\lambda)恰好有n个跳跃点， 则\Gamma_n正定而\Gamma_{n+1}退化。 如果F(\lambda)有无穷个跳跃点， 则对任意n \geq 1，\Gamma_n都是正定的。
证明： \forall \boldsymbol b = (b_1, \dots, b_n)^T, \lambda的函数 g(\lambda) = \sum_{k=1}^n b_k e^{i k \lambda} 是函数 h(z) = \sum_{k=1}^n b_k z^k = z \sum_{j=0}^{n-1} b_{j+1} z^j 在z = e^{i\lambda}的值， 所以g(\lambda)至多有n-1个零点。 当F(\lambda)的跳跃点个数\geq n时， \begin{aligned} \boldsymbol b^T \Gamma_n \boldsymbol b =& \int_{-\pi}^{\pi} \left| \sum_{k=1}^n b_k e^{i k \lambda} \right|^2 \, d F(\lambda) \\ =& \int_{-\pi}^{\pi} |g(\lambda)|^2 \, d F(\lambda) \end{aligned} 关于d F(\lambda)的积分等于跳跃高度乘以跳跃点处被积函数然后求和， 这里被积函数只有至多n-1个零点而跳跃点有n个以上， 所以求和至少有一项非零，故积分为正值， 即\Gamma_n正定。
如果G(\lambda)恰好有如下的n个跳跃点： -\pi < \lambda_1 < \dots < \lambda_n \leq \pi 定义复数b_0, b_1, \dots, b_n为 \sum_{k=0}^n b_k e^{i k \lambda} = \prod_{j=1}^n (1 - e^{i (\lambda - \lambda_j)}) 则b_0 = 1， 令\boldsymbol b = (b_0, b_1, \dots, b_n)^T \neq \boldsymbol 0， 当G(\cdot)是n/2个频率的离散谱序列时， 因为频率是相反数成对出现的， 所以\boldsymbol b为实向量。 对\boldsymbol b有 \boldsymbol b^* \Gamma_{n+1} \boldsymbol b = \int_{-\pi}^{\pi} \left| \sum_{k=0}^n b_k e^{i k \lambda} \right|^2 \, d F(\lambda) 因为函数\sum_{k=0}^n b_k e^{i k \lambda}在F(\lambda)的n个跳跃点处都等于零， 所以上述积分为零， 如果\boldsymbol b为实向量， 则\Gamma_{n+1}退化。
如果\boldsymbol b为复向量， 对零均值平稳列\{ X_t \}有 \text{Var}(\sum_{j=0}^n b_j X_j) = E \left| \sum_{j=0}^n b_j X_j \right|^2 = \boldsymbol b^* \Gamma_{n+1} \boldsymbol b = 0 取\boldsymbol a = \text{Re}(\boldsymbol b)， 则\boldsymbol a^T \Gamma_{n+1} \boldsymbol a = 0， 且a_0 = b_0 = 1故\boldsymbol a \neq \boldsymbol 0， \Gamma_{n+1}退化。 定理证毕。
○○○○○○
9.5 时间序列的可完全预测性
有限个频率的离散谱序列的轨道具有周期性， 可以用有限个历史值的线性组合无误差地预报整个序列。
对于方差有限的随机变量Y_1, Y_2,\cdots,Y_n, 如果有不全为零的常数b_1,\dots,b_n使得 \text{Var} \large(\sum_{j=1}^n b_j Y_{j} \large) =0, 则称随机变量Y_1, Y_2,\cdots,Y_n 是线性相关的, 否则称为线性无关的.
线性相关时, 存在常数b_0使得\sum_{j=1}^n b_j Y_{j} = b_0
a.s.成立.
并且当b_n\neq 0时,
Y_n可以由Y_1, Y_2,\cdots,Y_{n-1}线性表示:
\begin{aligned}
Y_n = a_0 + a_1 Y_{n-1} + \dots + a_{n-1} Y_1
\end{aligned}
这时我们称Y_n可以由Y_1, Y_2,\cdots,Y_{n-1}
完全线性预测.
对于平稳序列\{X_t\}，X_{t-1}, \dots, X_{t-n}的一个带截距的线性组合为 b_1 X_{t-1} + \dots b_n X_{t-n} - b_0，这n个变量线性无关当且仅当 \begin{aligned} & \text{Var}(\sum_{j=1}^n b_j X_{t-j} - b_0) = \text{Var}(\sum_{j=1}^n b_j X_{t-j}) \\ =& \boldsymbol{b} \Gamma_n \boldsymbol{b} > 0 \end{aligned} 即\Gamma_n正定。
反之，若\Gamma_n正定而\Gamma_{n+1}不满秩， 则X_t可以被X_{t-1}, \dots, X_{t-n}完全线性预测。
线性平稳列不能完全线性预测。
有限个频率成分的离散谱序列可完全线性预测。