9 AR(\(p\))序列的谱密度和Yule-Walker方程

9.1 AR(\(p\))序列的谱密度

9.1.1 AR(\(p\))序列的自协方差

因为AR(\(p\))的平稳解为 \[ X_t = A^{-1}(\mathscr B) \varepsilon_t = \sum_{j=0}^\infty \psi_j \varepsilon_{t-j} \] 由线性平稳列性质知\(\{X_t\}\)为零均值，自协方差函数为 \[\begin{align} \gamma_k = E(X_{t+k} X_t) = \sigma^2 \sum_{j=0}^\infty \psi_j \psi_{j+k}, \quad k=0, 1, \dots \tag{9.1} \end{align}\]

设\(1<\rho<\min\{|z_j|\}\)，则\(\psi_j = o(\rho^{-j})\), 有 \[\begin{align} |\gamma_k| \leq & \sigma^2 \sum_{j=0}^\infty |\psi_j| \cdot |\psi_{j+k}| \leq \sigma^2 (\sum_{j=0}^\infty |\psi_j|) (\sum_{l=k}^\infty |\psi_l|) \\ \leq & c_0 \sum_{l=k}^\infty \rho^{-l} \leq c_1 \rho^{-k} \tag{9.2} \end{align}\] 即\(\{\gamma_k\}\)负指数衰减。

\(\{X_t\}\)序列前后的相关减小很快， 称为时间序列的短记忆性。 征根离单位圆越远\(\{\gamma_k\}\)衰减越快。

9.1.2 AR(\(p\))的谱密度

由线性平稳列的谱密度公式得平稳解的谱密度 \[\begin{aligned} f(\lambda) = \frac{\sigma^2}{2\pi} \left| \sum_{j=0}^\infty \psi_j e^{ij\lambda} \right|^2 \end{aligned}\] 而\(\sum \psi_j z^j = 1/A(z)\)所以 \[\begin{align} f(\lambda) = \frac{\sigma^2}{2\pi} \frac{1}{\left| A(e^{i\lambda}) \right|^2} \tag{9.3} \end{align}\] \(f(\lambda)\)是一个恒正的偶函数。 如果\(A(z)\)有靠近单位圆的根\(\rho_j e^{i\lambda_j}\)则 \(|A(e^{i\lambda_j})|\)会接近零， 造成谱密度在\(\lambda=\lambda_j\)处有一个峰值。

9.1.3 谱密度的自协方差函数反演公式

谱密度的定义是满足 \[\begin{aligned} \gamma_k = \int_{-\pi}^\pi e^{ik\lambda} f(\lambda) d\lambda \end{aligned}\] 的非负可积函数。上式是一个Fourier级数系数的公式(差一个常数)。

在\(\{\gamma_k\}\)满足一定条件下\(f(\lambda)\)必存在， 且可表成\(\{\gamma_k\}\)的Fourier级数。

定理9.1 如果平稳序列\(\{X_t\}\)的自协方差函数\(\{\gamma_k\}\) 绝对可和: \(\sum|\gamma_k|<\infty\), 则\(\{X_t\}\)有谱密度 \[\begin{align} f(\lambda)=\frac{1}{2\pi} \sum_{k=-\infty}^\infty \gamma_k e^{-ik\lambda}. \tag{9.4} \end{align}\]

由于谱密度是实值函数, 所以(9.4)还可以写成 \[ f(\lambda)=\frac{1}{2\pi}\sum_{k=-\infty}^\infty \gamma_k \cos(k\lambda) = \frac{1}{2\pi} \left[ \gamma_0 + 2\sum_{k=1}^\infty \gamma_k \cos(k\lambda)\right]. \]

证明: 因为\(\{\gamma_k\}\)绝对可和所以(9.4)右边绝对一致收敛, \(f(\lambda)\)连续。 于是积分与级数可交换： \[\begin{aligned} \int_{-\pi}^\pi f(\lambda) e^{ij\lambda}\ d\lambda =\frac{1}{2\pi}\sum_{k=-\infty}^\infty \gamma_k \int_{-\pi}^\pi e^{-i(k-j)\lambda} \ d\lambda =\gamma_j. \end{aligned}\]

还要验证\(f(\lambda)\)非负。 若\(X_1, \dots, X_N\)为\(\{X_t\}\)的观测值， \[\begin{aligned} I_N(\lambda) = \frac{1}{2\pi N} \left| \sum_{j=1}^N X_j e^{ij\lambda} \right|^2, \quad \lambda \in [-\pi,\pi] \end{aligned}\] 称为\(X_1,\ldots, X_N\)的周期图。 令\(f_N(\lambda) = E I_N(\lambda)\), 则\(f_N(\lambda) \geq 0\)，于是 \[\begin{aligned} 0 \leq& f_N(\lambda) = \frac{1}{2\pi N} \sum_{k=1}^N \sum_{j=1}^N \gamma_{k-j} e^{-i(k-j)\lambda} \\ =& \frac{1}{2\pi N} \sum_{m=1-N}^{N-1} (N-|m|)\gamma_{m} e^{-im\lambda} \\ =& \frac{1}{2\pi} \sum_{m=1-N}^{N-1}\gamma_{m} e^{-im\lambda} -\frac{1}{2\pi N} \sum_{m=1-N}^{N-1}|m|\gamma_{m} e^{-im\lambda}. \end{aligned} \tag{*} \] 由Kronecker引理知后一项趋于0, 于是 \[\begin{aligned} f(\lambda) = \lim_{N\to \infty} f_N(\lambda) \geq 0. \end{aligned}\]

○○○○○○

附注：(*)式的二重求和的简化

\[\begin{aligned} & \frac{1}{2\pi N} \sum_{k=1}^N \sum_{j=1}^N \gamma_{k-j} e^{-i(k-j)\lambda} \\ =& \frac{1}{2\pi N} \sum_{k=1}^N \sum_{m=k-N}^{k-1} \gamma_{m} e^{-im\lambda} \quad (\text{令}m = k-j) \end{aligned}\] 交换\(m\)与\(k\)的求和次序。因为关于\(m\)的条件为 \[\begin{aligned} k - N \leq m \leq k - 1 \end{aligned}\] 所以\(k \leq m+N\), \(k \geq m+1\)， 求和变为 \[\begin{aligned} \frac{1}{2\pi N} \sum_{m=1-N}^{N-1} \sum_{k=\max(m+1,1)}^{k=\min(m+N,N)} \gamma_{m} e^{-im\lambda} \end{aligned}\] 因为\(m \geq 0\)时\(k\)的求和从\(m+1\)到\(N\)有\(N-m\)项， \(m<0\)时\(k\)的求和从\(1\)到\(m+N=N-|m|\)有\(N-|m|\)项，所以求和变为 \[\begin{aligned} \frac{1}{2\pi N} \sum_{m=1-N}^{N-1} (N - |m|) \gamma_{m} e^{-im\lambda} \end{aligned}\]

附注：Kronecker引理： 设复数或实数级数\(\sum_{j=1}^n x_j\)收敛， 非负实数列\({b_n}\)单调不减趋于\(+\infty\)， 则 \[ \lim_{n\to\infty} \frac{1}{b_n} \sum_{j=1}^n b_j x_j = 0 . \]

见B.2。

○○○○○○

推论9.1 AR(\(p\))的平稳解序列\(\{X_t\}\)有谱密度 \[\begin{aligned} f(\lambda)=\frac{1}{2\pi}\sum_{k=-\infty}^\infty \gamma_k e^{-ik\lambda} =\frac{\sigma^2}{2\pi}\frac{1}{|A(e^{i\lambda})|^2}. \end{aligned}\]

9.2 Yule-Walker方程

9.2.1 白噪声列与平稳解的关系

\(A(\mathscr B) X_t = \varepsilon_t\)的平稳解为 \[\begin{aligned} X_t = A^{-1}(\mathscr B) \varepsilon_t = \sum_{j=0}^\infty \psi_j \varepsilon_{t-j} \end{aligned}\] 对\(k \geq 1\)由控制收敛定理或内积的连续性得 \[\begin{aligned} E(X_t \varepsilon_{t+k} ) = \sum_{j=0}^\infty \psi_j E(\varepsilon_{t-j} \varepsilon_{t+k}) = 0 \end{aligned}\] 即\(X_t\)与未来的输入不相关。

如果\(\{\varepsilon_t\}\)是独立白噪声则\(X_t\)与未来的输入独立。

9.2.2 AR序列的等价定义

设\(a_1, \dots, a_p\)为实数， \(\{ \varepsilon_t \}\)为WN(\(0, \sigma^2\)), 若平稳列\(\{ X_t, t \in \mathbb Z \}\)满足 \[ X_t = a_1 X_{t-1} + \dots + a_p x_{t-p} + \varepsilon_t, \ t \in \mathbb Z, \] 且\(\varepsilon_{t+k}, k=1,2,\dots\)与\(\{ X_s: s \leq t \}\)互不相关， 则称\(\{ X_t \}\)为AR(\(p\))序列。

可以看出，两个定义是等价的。 由前面的推导可知原始定义可以导出等价定义中的性质。 反之，如果等价定义成立， \(A(z)=1 - a_1 z - \dots - a_p z^p\)一定满足最小相位性。

9.2.3 Yule-Walker方程

在AR(\(p\))模型 \[ X_t = a_1 X_{t-1} + \dots + a_p X_{t-p} + \varepsilon_{t-p} \] 两边同时乘以\(X_{t-k}\)(\(k \geq 1\))后取期望， 有 \[ E(X_t X_{t-k}) = a_1 E(X_{t-1} X_{t-k}) + \dots + a_p E(X_{t-p} X_{t-k}) + E(\varepsilon_t X_{t-k}) \] 即有 \[ \gamma_k = a_1 \gamma_{k-1} + \dots + a_p \gamma_{k-p} + 0 \]

所以，对\(k \geq 1\)，有递推式 \[ \gamma_k = a_1 \gamma_{k-1} + \dots + a_p \gamma_{k-p} . \]

\(k \geq 1\)时，\(\gamma_k\)满足齐次线性差分方程 \[ A(\mathscr B) \gamma_k = 0 . \]

将\(k=1,2,\dots,p\)的方程写成 \[\begin{aligned} \gamma_1 =& a_1 \gamma_0 + a_2 \gamma_1 + \dots + a_p \gamma_{p-1} \\ \gamma_2 =& a_1 \gamma_1 + a_2 \gamma_0 + \dots + a_p \gamma_{p-2} \\ \vdots =& \vdots \\ \gamma_p =& a_1 \gamma_{p-1} + a_2 \gamma_{p-2} + \dots + a_p \gamma_0 \end{aligned}\] 写成矩阵形式： \[\begin{aligned} \left(\begin{matrix} \gamma_1 \\ \gamma_2 \\ \gamma_3 \\ \vdots \\ \gamma_p \end{matrix}\right) =& \left(\begin{matrix} \gamma_0 & \gamma_1 & \cdots & \gamma_{p-1} \\ \gamma_1 & \gamma_0 & \cdots & \gamma_{p-2} \\ \gamma_2 & \gamma_1 & \cdots & \gamma_{p-3} \\ \vdots & \ddots & \ddots & \vdots \\ \gamma_{p-1} & \gamma_{p-2} & \cdots & \gamma_0 \end{matrix}\right) \left(\begin{matrix} a_1 \\ a_2 \\ \vdots \\ a_p \end{matrix}\right) \end{aligned}\] 记 \[\begin{aligned} \boldsymbol{\gamma}_p =& \left(\begin{matrix} \gamma_1 \\ \gamma_2 \\ \gamma_3 \\ \vdots \\ \gamma_p \end{matrix}\right), \ \Gamma_p = \left(\begin{matrix} \gamma_0 & \gamma_1 & \cdots & \gamma_{p-1} \\ \gamma_1 & \gamma_0 & \cdots & \gamma_{p-2} \\ \gamma_2 & \gamma_1 & \cdots & \gamma_{p-3} \\ \vdots & \ddots & \ddots & \vdots \\ \gamma_{p-1} & \gamma_{p-2} & \cdots & \gamma_0 \end{matrix}\right), \ \boldsymbol{a}_p = \left(\begin{matrix} a_1 \\ a_2 \\ \vdots \\ a_p \end{matrix}\right) \end{aligned}\] 有 \[\begin{aligned} & \Gamma_p \boldsymbol{a}_p = \boldsymbol{\gamma}_p \end{aligned}\]

对\(\gamma_0\)， \[\begin{aligned} E[X_t \varepsilon_t] =& E[ (a_1 X_{t-1} + \dots + a_p X_{t-p} + \varepsilon_t) \varepsilon_t ] \\ =& 0 + \sigma^2 = \sigma^2 . \end{aligned}\] 在\(X_t = a_1 X_{t-1} + \dots + a_p X_{t-p} + \varepsilon_t\)两边同时乘以\(X_t\)后取期望，得
\[\begin{aligned} \gamma_0 =& a_1 \gamma_1 + \dots + a_p \gamma_p + E[X_t \varepsilon_t] \\ =& a_1 \gamma_1 + \dots + a_p \gamma_p + \sigma^2 . \end{aligned}\]

上式另一证明为： \[\begin{aligned} \gamma_0 =& E X_t^2 = E \left( \sum_{j=1}^p a_j X_{t-j} + \varepsilon_t \right)^2 \\ =& E \left(\sum_{j=1}^p a_j X_{t-j}\right)^2 + E \varepsilon_t^2 \\ =& \boldsymbol{a}_p^T \Gamma_p \boldsymbol{a}_p + \sigma^2 \\ =& \boldsymbol{a}_p^T \boldsymbol{\gamma}_p + \sigma^2 \\ =& a_1 \gamma_1 + a_2\gamma_2 + \dots + a_p \gamma_p + \sigma^2 \end{aligned}\]

于是 \[\begin{aligned} \gamma_k =& a_1 \gamma_{k-1} + a_2\gamma_{k-2} + \dots + a_p \gamma_{k-p}, \ k \geq 1 \\ \gamma_0 =& a_1 \gamma_1 + a_2\gamma_2 + \dots + a_p \gamma_p + \sigma^2 . \end{aligned}\] 用推移算子写成 \[\begin{aligned} A(\mathscr B) \gamma_k =& 0, \ k \geq 1 \\ A(\mathscr B) \gamma_0 =& \sigma^2 . \end{aligned}\]

对\(n \geq p\)，令 \[\begin{aligned} \boldsymbol{\gamma}_n =& \left(\begin{matrix} \gamma_1 \\ \gamma_2 \\ \gamma_3 \\ \vdots \\ \gamma_n \end{matrix}\right), \ \boldsymbol{a}_n = \left(\begin{matrix} a_1 \\ a_2 \\ \vdots \\ a_p \\ 0 \\ \vdots \\ 0 \end{matrix}\right) \end{aligned}\] 仍有

\[\begin{align} \Gamma_n \boldsymbol{a}_n = \boldsymbol{\gamma}_n \tag{9.5} \end{align}\] 事实上，对\(n \geq p\), 把\(X_t, X_{t+1}, \dots, X_{t+n-1}\)的递推式写成矩阵形式得 \[\begin{align} & \left( \begin{array}{l} X_t\\ X_{t+1}\\ \vdots \\ X_{t+n-1} \end{array} \right) \\ = & \left( \begin{array}{llll} X_{t-1} & X_{t-2} & \dots &X_{t-n}\\ X_{t} & X_{t-1} & \dots &X_{t+1-n}\\ \vdots & \vdots & & \vdots \\ X_{t+n-2} & X_{t+n-3} & \dots & X_{t-1}\\ \end{array} \right) \boldsymbol{a}_n + \left( \begin{array}{l} \varepsilon_t\\ \varepsilon_{t+1}\\ \vdots \\ \varepsilon_{t+n-1} \end{array} \right), \tag{9.6} \end{align}\] 在(9.6)两边同时乘上\(X_{t-1}\)后取数学期望, 利用\(X_t\)与未来输入的不相关性就可以证明(9.5)式。

定理9.2 (Yule-Walker方程) AR\((p)\)序列的自协方差函数满足 \[\begin{align} \boldsymbol{\gamma}_n = \Gamma_n\boldsymbol{a}_n, \quad \gamma_0 = \boldsymbol{\gamma}_n^T \boldsymbol{a}_n + \sigma^2, \quad n\geq p, \tag{9.7} \end{align}\] 即 \[\begin{align} \gamma_k =& a_1 \gamma_{k-1} + a_2 \gamma_{k-2} + \dots + a_p \gamma_{k-p}, \quad k\geq 1 \tag{9.8} \\ A(\mathscr B) \gamma_k =& 0, \quad k \geq 1 \\ A(\mathscr B) \gamma_0 =& \gamma_0 - a_1 \gamma_1 - a_2 \gamma_2 - \dots - a_p \gamma_p = \sigma^2 \tag{9.9} \end{align}\] 特别地，当\(n=p\)时 \[\begin{align} & \Gamma_p \left(\begin{array}{c} a_1 \\ a_2 \\ \dots \\ a_p \end{array} \right) = \left(\begin{array}{c} \gamma_1 \\ \gamma_2 \\ \dots \\ \gamma_p \end{array} \right) \tag{9.10} \end{align}\]

记\(\phi_0=1, \phi_1 = -a_1, \dots, \phi_p=-a_p\), 则\(A(z) = \sum_{j=0}^p \phi_j z^j\)， AR模型可写成\(\sum_{j=0}^p \phi_j X_{t-j} = \varepsilon_t\)。

Yule-Walker方程可写成 \[\begin{aligned} \left(\begin{array}{cccc} \gamma_0 & \gamma_1 & \cdots & \gamma_p \\ \gamma_1 & \gamma_0 & \cdots & \gamma_{p-1} \\ \vdots & \vdots & & \vdots \\ \gamma_p & \gamma_{p-1} & \cdots & \gamma_0 \end{array}\right) \left(\begin{array}{c} \phi_0 \\ \phi_1 \\ \vdots \\ \phi_p \end{array}\right) = \left(\begin{array}{c} \sigma^2 \\ 0 \\ \vdots \\ 0 \end{array}\right) \end{aligned}\]

9.3 自协方差函数的周期性

对\(k<0\)定义\(\psi_k=0\)。 ::: {.corollary #arspecyw-ywzz} AR(\(p\))序列的自协方差函数\(\{\gamma_k\}\)满足和 AR(\(p\))模型\(A(\mathscr B) X_t = \varepsilon_t\)相应的差分方程 \[\begin{aligned} \gamma_k - (a_1 \gamma_{k-1} + a_2 \gamma_{k-2} + \dots + a_p \gamma_{k-p}) = \sigma^2 \psi_{-k}, \quad k \in \mathbb Z \end{aligned}\] :::

证明: \(k \geq 0\)时即定理结论。对\(k<0\)， \[\begin{aligned} & \gamma_k - (a_1 \gamma_{k-1} + a_2 \gamma_{k-2} + \dots + a_p \gamma_{k-p}) \\ =& E\left[ X_{t-k} ( X_t - \sum_{j=1}^p a_j X_{t-j} ) \right] \\ =& E(X_{t-k} \varepsilon_t) = E \left[ \sum_{j=0}^\infty \psi_j \varepsilon_{t-k-j} \varepsilon_t \right] = \sigma^2 \psi_{-k} \end{aligned}\]

○○○○○○

设\(A(z)\)有\(p\)个互异根\(z_j=\rho_j e^{i\lambda_j}, j=1,\dots,p\)， 可以证明（略） \[\begin{align} \gamma_t =& A^{-1}(\mathscr B) \sigma^2 \psi_{-t} \\ =& \sigma^2 \sum_{j=1}^p c_j A^{-1}(z_j^{-1}) z_j^{-t} \\ =& \sigma^2 \sum_{j=1}^p A_j \rho_j^{-t} \cos(\lambda_j t + \theta_j), \quad t \geq 0 \tag{9.11} \end{align}\] 可见如果\(\{z_j\}\)中有靠近单位圆的复根则\(\{\gamma_k\}\) 的衰减振荡特性会显现出来。

例9.1 AR(4)模型1: \[\begin{aligned} z_1,z_2 = 1.09e^{\pm i\pi/3}, \quad z_3,z_4 = 1.098e^{\pm i 2\pi/3} \end{aligned}\] 周期为\(2\pi/(\pi/3)=6\)和\(2\pi/(2\pi/3)=3\)。

AR(4)模型2: \[\begin{aligned} z_1,z_2 = 1.264e^{\pm i\pi/3}, \quad z_3,z_4 = 1.273e^{\pm i 2\pi/3} \end{aligned}\]

AR(4)模型3: \[\begin{aligned} z_1,z_2 = 1.635e^{\pm i\pi/3}, \quad z_3,z_4 = 1.647e^{\pm i 2\pi/3} \end{aligned}\]

程序演示:

library(polynom)
demo.ar.roots <- function(){
  n <- 1024
  rtlis <- list(c(complex(mod=1.09, arg=pi/3*c(1,-1)),
                  complex(mod=1.098, arg=pi*2/3*c(1,-1))),
                c(complex(mod=1.264, arg=pi/3*c(1,-1)),
                  complex(mod=1.273, arg=pi*2/3*c(1,-1))),
                c(complex(mod=1.635, arg=pi/3*c(1,-1)),
                  complex(mod=1.647, arg=pi*2/3*c(1,-1))),
                complex(mod=1.02, arg=pi/6*c(1,-1)),
                complex(mod=1.02, arg=pi/2*c(1,-1)),
                c(complex(mod=1.05, arg=pi/6*c(1,-1)),
                  complex(mod=1.05, arg=pi/2*c(1,-1))))
  tits <- c("AR(4): 1.09exp(+-i pi/3), 1.098exp(+-i 2pi/3)",
            "AR(4): 1.264exp(+-i pi/3), 1.273exp(+-i 2pi/3)",
            "AR(4): 1.635exp(+-i pi/3), 1.647exp(+-i 2pi/3)",
            "AR(2): mod=1.02 arg=+-pi/6",
            "AR(2): mod=1.02 arg=+-pi/2",
            "AR(4): mod=1.05 arg=+-pi/6,+-pi/2")
  tits <- c(
    expression(paste("AR(4):", 
                     list(1.09*e^{phantom(.) %+-% i*frac(pi,3)}, 
                          1.098*e^{phantom(.) %+-% i*frac(2*pi,3)}) )),
    expression(paste("AR(4):", 
                     list(1.264*e^{phantom(.) %+-% i*frac(pi,3)}, 
                          1.273*e^{phantom(.) %+-% i*frac(2*pi,3)}) )),
    expression(paste("AR(4):", 
                     list(1.635*e^{phantom(.) %+-% i*frac(pi,3)}, 
                          1.647*e^{phantom(.) %+-% i*frac(2*pi,3)}) )),
    expression(paste("AR(2):", 
                     list(1.02*e^{phantom(.) %+-% i*frac(pi,6)}) )),
    expression(paste("AR(2):", 
                     list(1.02*e^{phantom(.) %+-% i*frac(pi,2)}) )),
    expression(paste("AR(4):", 
                     list(1.05*e^{phantom(.) %+-% i*frac(pi,6)}, 
                          1.05*e^{phantom(.) %+-% i*frac(pi,2)}) ))
    )
  oldpar <- par(mfrow=c(3,1), mar=c(2,2,0,0), 
                mgp=c(2, 0.5, 0), oma=c(0,0,2,0))
  on.exit(par(oldpar))
  for(ii in seq(along=rtlis)){
    rt = rtlis[[ii]]
    y <- ar.gen(n, rt, sigma=1.0, by.roots=TRUE,
                plot.it=FALSE)
    plot(window(y, 1, 60))
    acf(y)
    ##spectrum(y, taper=0.2)
    ar.true.spectrum(attr(y, "a"), title="")
    mtext(tits[ii], side=3, outer=TRUE)
  }
}
demo.ar.roots()

## Warning in polynomial(p): 强制改变时丢弃了虚数部分

## Warning in polynomial(p): 强制改变时丢弃了虚数部分

## Warning in polynomial(p): 强制改变时丢弃了虚数部分

9.4 自协方差函数的正定性

AR(\(p\))平稳解唯一故自协方差函数可被自回归系数和白噪声方差唯一决定。 反之， 若\(\Gamma_p\)正定则根据Yule-Walker方程可以从 \(\gamma_0, \gamma_1, \dots, \gamma_p\)解出\(a_1, \dots, a_p, \sigma^2\): \[\begin{align} \boldsymbol{a}_p = \Gamma_p^{-1} \boldsymbol{\gamma}_p, \quad \sigma^2 = \gamma_0 - \boldsymbol{\gamma}_p^T \Gamma_p^{-1} \boldsymbol{\gamma}_p. \tag{9.12} \end{align}\]

定理9.3 设\(\Gamma_n\)是实值平稳序列\(\{X_t\}\)的\(n\)阶自协方差矩阵, \(\gamma_0>0\)。

(1) 如果\(\{X_t\}\)的谱密度\(f(\lambda)\)存在, 则对\(n\geq 1\)，\(\Gamma_n\)正定；

(2) 如果\(\lim_{k \to \infty} \gamma_k = 0\)， 则对\(n \geq 1\)，\(\Gamma_n\)正定。

引理9.1 对实平稳列\(\{X_t\}\)，设其自协方差阵为\(\Gamma_n\), \(n \in \mathbb N\)； 设其谱函数为\(F(\lambda)\)。 \(\forall \boldsymbol{b}=(b_1, \dots, b_n) \in \mathbb C^n\)有 \[\begin{aligned} \boldsymbol{b}^* \Gamma_n \boldsymbol{b} = \int_{-\pi}^\pi \left| \sum_{j=1}^n b_j e^{ij\lambda} \right|^2 dF(\lambda) \end{aligned}\] 若\(\{X_t\}\)有谱密度\(f(\lambda)\)则 \[\begin{aligned} \boldsymbol{b}^* \Gamma_n \boldsymbol{b} = \int_{-\pi}^\pi \left| \sum_{j=1}^n b_j e^{ij\lambda} \right|^2 f(\lambda) d\lambda \end{aligned}\]

引理证明

\[\begin{aligned} & \boldsymbol{b}^* \Gamma_n \boldsymbol{b} = \sum_{j=1}^n \sum_{k=1}^n \bar b_j b_k \gamma_{k-j} \\ =& \sum_{j=1}^n \sum_{k=1}^n \bar b_j b_k \int_{-\pi}^\pi e^{i(k-j)\lambda} dF(\lambda) \\ =& \int_{-\pi}^\pi \sum_{j=1}^n \sum_{k=1}^n \bar b_j b_k e^{-ij\lambda} e^{ik\lambda} dF(\lambda) \\ =& \int_{-\pi}^\pi \overline{\left( \sum_{j=1}^n b_j e^{ij\lambda} \right)} \left( \sum_{k=1}^n b_k e^{ik\lambda} \right) \,dF(\lambda)\\ =& \int_{-\pi}^\pi \left| \sum_{j=1}^n b_j e^{ij\lambda} \right|^2 dF(\lambda) \end{aligned}\]

定理证明

(1) 对\(\boldsymbol{b} = (b_1, \dots, b_n)^T\)， \(\sum_{j=1}^n b_j z^{j-1}\) 至多有\(n-1\)个零点。\(\gamma_0 = \int_{-\pi}^\pi f(\lambda) d\lambda > 0\), 于是 \[\begin{aligned} \boldsymbol{b}^T \Gamma_n \boldsymbol{b} = \int_{-\pi}^\pi \left| \sum_{j=1}^n b_j e^{ij\lambda} \right|^2 f(\lambda) d \lambda > 0 \end{aligned}\]

(2) 用反证法。 设\(\Gamma_n\)正定, \(\det(\Gamma_{n+1})=0\)和\(EX_t=0\) (非零均值情况只要减去均值).
定义 \[ \boldsymbol{X}_n=(X_1,X_{2},\dots, X_n)^T \] 对任何实向量 \(\boldsymbol{b}=(b_1,b_2,\dots,b_n)^T \neq 0\) 有 \[ E(\boldsymbol{b}^T \boldsymbol{X}_n)^2 =\boldsymbol{b}^T \Gamma_n \boldsymbol{b} >0, \] 且由\(|\Gamma_{n+1}|=0\)知存在 \(\boldsymbol{a} =(a_1,a_2,\dots,a_{n+1})^T \neq 0\), \(a_{n+1}\neq 0\)使得 \[ E(\boldsymbol{a}^T \boldsymbol{X}_{n+1})^2 = \boldsymbol{a} ^T \Gamma_{n+1} \boldsymbol{a} =0. \] 于是 \[ \boldsymbol{a}^T \boldsymbol{X}_{n+1} = a_1 X_1 + a_2 X_2 + \dots + a_{n+1} X_{n+1}=0 \] a.s.成立, \(X_{n+1}\)可以由\(\boldsymbol{X}_n\)线性表示: \[\begin{aligned} X_{n+1} = -\frac{a_n}{a_{n+1}} X_{n} - \frac{a_{n-1}}{a_{n+1}} X_{n-1} - \dots - \frac{a_1}{a_{n+1}} X_{1}, \quad \text{a.s.}, \end{aligned}\]

利用\(\{X_t\}\)的平稳性知道 \[\begin{aligned} X_t = -\frac{a_n}{a_{n+1}} X_{t-1} - \frac{a_{n-1}}{a_{n+1}} X_{t-2} - \dots - \frac{a_1}{a_{n+1}} X_{t-n}, \quad \text{a.s.}, \quad t \in \mathbb Z \end{aligned}\]

递推知对任何\(k\geq 1\), \(X_{n+k}\) 可以由\(X_1,X_2,\dots,X_n\)线性表示, 即有实向量 \(\boldsymbol{\alpha} \stackrel{\triangle}{=} \boldsymbol{\alpha}^{(k)} \stackrel{\triangle}{=} (\alpha_1^{(k)}, \dots, \alpha_n^{(k)})^T\) 使得 \[ X_{n+k}=(\boldsymbol{\alpha}^{(k)})^T \boldsymbol{X}. \]

\(X_{n+k}\)被\(\boldsymbol{X}\)线性表示， 说明\(X_{n+k}\)与\(X_1,\dots,X_n\)有强的相关， 而定理假设是\(\gamma_k \to 0\)， 又说明\(X_{n+k}\)与\(X_1,\dots,X_n\)的相关性要趋于零， 这就会有矛盾，下面把矛盾严格表述。

用 \(0 < \lambda_1 \leq \lambda_2 \dots \leq \lambda_n\) 表示\(\Gamma_n\)的特征值, 则有正交矩阵 \(T\)使得 \[ T\Gamma_n T^T = \text{diag}(\lambda_1, \lambda_2,\dots,\lambda_n). \]
用\(|\boldsymbol{\alpha}^{(k)}|\) 表示\(\boldsymbol{\alpha}^{(k)}\)的欧氏模, 则有 \[\begin{aligned} \gamma_0 =& E X_{n+k}^2 = E((\boldsymbol{\alpha}^{(k)})^T \boldsymbol{X})^2 = (\boldsymbol{\alpha}^{(k)})^T \Gamma_n \boldsymbol{\alpha}^{(k)} \\ =& ( (\boldsymbol{\alpha}^{(k)})^T T^T) (T \Gamma_n T^T) (T \boldsymbol{\alpha}^{(k)} )\\ \geq& \lambda_1 (T \boldsymbol{\alpha}^{(k)})^T (T \boldsymbol{\alpha}^{(k)} ) = \lambda_1 |\boldsymbol{\alpha}^{(k)}|^2. \end{aligned}\] 即有\(|\boldsymbol{\alpha}^{(k)} | \leq \sqrt{\gamma_0/\lambda_1} < \infty\).

另一方面 \[\begin{aligned} \gamma_0 =& E( (\boldsymbol{\alpha}^{(k)})^T \boldsymbol{X} \cdot X_{n+k}) = (\boldsymbol{\alpha}^{(k)})^T E(\boldsymbol{X} X_{n+k} ) \\ =& (\boldsymbol{\alpha}^{(k)})^T (\gamma_{n+k-1}, \gamma_{n+k-2}, \dots,\gamma_{k})^T\\ \leq & |\boldsymbol{\alpha}^{(k)}| \left(\sum_{j=0}^{n-1}\gamma_{j+k}^2\right)^{1/2}\\ \leq & (\gamma_0/\lambda_1)^{1/2} \left(\sum_{j=0}^{n-1}\gamma_{k+j}^2\right)^{1/2}\\ \to & 0. \quad (\text{当}k\to \infty) \end{aligned}\] 这与\(\gamma_0>0\)矛盾, 故 \(\det(\Gamma_{n+1})=0\)不成立.

○○○○○○

推论9.2 系数平方可和的线性平稳序列的自协方差阵总是正定的。

对平稳列\(\{X_n \}\)的自协方差函数\(\{\gamma_j, j \in \mathbb Z\}\)， 如果\(\{X_n \}\)自协方差阵总是正定的， 则称自协方差函数\(\{\gamma_j, j \in \mathbb Z\}\)是正定序列。

定理9.4 设实平稳列\(\{ X_t \}\)的谱函数\(F(\lambda)\)是阶梯函数。 如果\(F(\lambda)\)恰好有\(n\)个跳跃点， 则\(\Gamma_n\)正定而\(\Gamma_{n+1}\)退化。 如果\(F(\lambda)\)有无穷个跳跃点， 则对任意\(n \geq 1\)，\(\Gamma_n\)都是正定的。

证明： \(\forall \boldsymbol b = (b_1, \dots, b_n)^T\), \(\lambda\)的函数 \[ g(\lambda) = \sum_{k=1}^n b_k e^{i k \lambda} \] 是函数 \[ h(z) = \sum_{k=1}^n b_k z^k = z \sum_{j=0}^{n-1} b_{j+1} z^j \] 在\(z = e^{i\lambda}\)的值， 所以\(g(\lambda)\)至多有\(n-1\)个零点。 当\(F(\lambda)\)的跳跃点个数\(\geq n\)时， \[\begin{aligned} \boldsymbol b^T \Gamma_n \boldsymbol b =& \int_{-\pi}^{\pi} \left| \sum_{k=1}^n b_k e^{i k \lambda} \right|^2 \, d F(\lambda) \\ =& \int_{-\pi}^{\pi} |g(\lambda)|^2 \, d F(\lambda) \end{aligned}\] 关于\(d F(\lambda)\)的积分等于跳跃高度乘以跳跃点处被积函数然后求和， 这里被积函数只有至多\(n-1\)个零点而跳跃点有\(n\)个以上， 所以求和至少有一项非零，故积分为正值， 即\(\Gamma_n\)正定。

如果\(G(\lambda)\)恰好有如下的\(n\)个跳跃点： \[ -\pi < \lambda_1 < \dots < \lambda_n \leq \pi \] 定义复数\(b_0, b_1, \dots, b_n\)为 \[ \sum_{k=0}^n b_k e^{i k \lambda} = \prod_{j=1}^n (1 - e^{i (\lambda - \lambda_j)}) \] 则\(b_0 = 1\)， 令\(\boldsymbol b = (b_0, b_1, \dots, b_n)^T \neq \boldsymbol 0\)， 当\(G(\cdot)\)是\(n/2\)个频率的离散谱序列时， 因为频率是相反数成对出现的， 所以\(\boldsymbol b\)为实向量。 对\(\boldsymbol b\)有 \[ \boldsymbol b^* \Gamma_{n+1} \boldsymbol b = \int_{-\pi}^{\pi} \left| \sum_{k=0}^n b_k e^{i k \lambda} \right|^2 \, d F(\lambda) \] 因为函数\(\sum_{k=0}^n b_k e^{i k \lambda}\)在\(F(\lambda)\)的\(n\)个跳跃点处都等于零， 所以上述积分为零， 如果\(\boldsymbol b\)为实向量， 则\(\Gamma_{n+1}\)退化。

如果\(\boldsymbol b\)为复向量， 对零均值平稳列\(\{ X_t \}\)有 \[ \text{Var}(\sum_{j=0}^n b_j X_j) = E \left| \sum_{j=0}^n b_j X_j \right|^2 = \boldsymbol b^* \Gamma_{n+1} \boldsymbol b = 0 \] 取\(\boldsymbol a = \text{Re}(\boldsymbol b)\)， 则\(\boldsymbol a^T \Gamma_{n+1} \boldsymbol a = 0\)， 且\(a_0 = b_0 = 1\)故\(\boldsymbol a \neq \boldsymbol 0\)， \(\Gamma_{n+1}\)退化。 定理证毕。

○○○○○○

9.5 时间序列的可完全预测性

有限个频率的离散谱序列的轨道具有周期性， 可以用有限个历史值的线性组合无误差地预报整个序列。

对于方差有限的随机变量\(Y_1, Y_2,\cdots,Y_n\), 如果有不全为零的常数\(b_1,\dots,b_n\)使得 \[ \text{Var} \large(\sum_{j=1}^n b_j Y_{j} \large) =0, \] 则称随机变量\(Y_1, Y_2,\cdots,Y_n\) 是线性相关的, 否则称为线性无关的.

线性相关时, 存在常数\(b_0\)使得\(\sum_{j=1}^n b_j Y_{j} = b_0\) a.s.成立.
并且当\(b_n\neq 0\)时, \(Y_n\)可以由\(Y_1, Y_2,\cdots,Y_{n-1}\)线性表示: \[\begin{aligned} Y_n = a_0 + a_1 Y_{n-1} + \dots + a_{n-1} Y_1 \end{aligned}\] 这时我们称\(Y_n\)可以由\(Y_1, Y_2,\cdots,Y_{n-1}\) 完全线性预测.

对于平稳序列\(\{X_t\}\)，\(X_{t-1}, \dots, X_{t-n}\)的一个带截距的线性组合为 \(b_1 X_{t-1} + \dots b_n X_{t-n} - b_0\)，这\(n\)个变量线性无关当且仅当 \[\begin{aligned} & \text{Var}(\sum_{j=1}^n b_j X_{t-j} - b_0) = \text{Var}(\sum_{j=1}^n b_j X_{t-j}) \\ =& \boldsymbol{b} \Gamma_n \boldsymbol{b} > 0 \end{aligned}\] 即\(\Gamma_n\)正定。

反之，若\(\Gamma_n\)正定而\(\Gamma_{n+1}\)不满秩， 则\(X_t\)可以被\(X_{t-1}, \dots, X_{t-n}\)完全线性预测。

线性平稳列不能完全线性预测。

有限个频率成分的离散谱序列可完全线性预测。