\[
\newcommand{\bs}{\boldsymbol}
\newcommand{\bsX}{\boldsymbol{X}}
\newcommand{\bf}{\mathbf}
\newcommand{\msc}{\mathscr}
\newcommand{\mca}{\mathcal}
\newcommand{\T}{\text{T}}
\newcommand{\rme}{\mathrm{e}}
\newcommand{\rmi}{\mathrm{i}}
\newcommand{\rmj}{\mathrm{j}}
\newcommand{\rmd}{\mathrm{d}}
\newcommand{\rmm}{\mathrm{m}}
\newcommand{\rmb}{\mathrm{b}}
\newcommand{\and}{\land}
\newcommand{\or}{\lor}
\newcommand{\exist}{\exists}
\newcommand{\sube}{\subseteq}
\newcommand{\lr}[3]{\left#1 #2 \right#3}
\newcommand{\intfy}{\int_{-\infty}^{+\infty}}
\newcommand{\sumfy}[1]{\sum_{#1=-\infty}^{+\infty}}
\newcommand{\vt}{\vartheta}
\newcommand{\ve}{\varepsilon}
\newcommand{\vp}{\varphi}
\newcommand{\Var}{\text{Var}}
\newcommand{\Cov}{\text{Cov}}
\newcommand{\edef}{\xlongequal{def}}
\newcommand{\prob}{\text{P}}
\newcommand{\Exp}{\text{E}}
\newcommand{\t}[1]{\text#1}
\newcommand{\N}{\mathbb{N}}
\newcommand{\Z}{\mathbb{Z}}
\newcommand{\Q}{\mathbb{Q}}
\newcommand{\R}{\mathbb{R}}
\newcommand{\C}{\mathbb{C}}
\newcommand{\versionofnewcommand}{\text{260125}}
\]
Correlation Function
有这样一个 idea, 从二阶的观点 (例如相关) 看问题, 要比一阶的观点效率高. 就像 Transformer 比 传统 CNN 效果好得多.
对于随机过程 \(X(t)\), 其 (自) 相关函数 (Auto-)correlation function 这样定义:
\[
R_\bs{X}(t,s)=E(\bs{X}(t)\bs{X}(s))
\]
它有三个显然的性质: 对称, 非负, Cauchy:
\[
\begin{aligned}
R_\bs{X}(t,s)&=R_\bs{X}(s,t)\\
R_\bs{X}(t,s)&\geq 0\\
|R_\bs{X}(t,s)|&\leq[R_\bs{X}(t,t)\cdot R_\bs{X}(s,s)]^{\frac{1}{2}}
\end{aligned}
\]
平稳性质 Stationality
A stochastic process is wide-sense stationary (W.S.S.), if :
\[
\begin{aligned}
&\text{I.}&
&E(\bs{X}(t))=\text{Const.}\\
&\text{II.}&
&R_\bs{X}(t+T,s+T)=R_\bs{X}(t,s),\ \forall T
\end{aligned}
\]
性质 \(\text{I}\) 既可以拓展为期望等于已知的函数, 也可以约束为期望等于零, 这都是不重要的.
性质 \(\text{II}\) 意味着 \(R_X\) 仅依赖于 \(t\) 和 \(s\) 的差值, 从而使得二元问题变为一元的. \(R_\bs{X}(t,s)\to R_\bs{X}(\tau)\).
W.S.S 的性质
显然, \(R_\bs{X}(\tau)\) 是个偶函数:
\[
R_\bs{X}(\tau)=R_\bs{X}(-\tau)
\]
且相关函数在零点非负:
\[
R_\bs{X}(0)=R_\bs{X}(t,t)=E(\bs{X}(t))^2\geq0
\]
另外, 由柯西不等式可以得出很重要的性质:
\[
|R_\bs{X}(\tau)|\leq R_\bs{X}(0)
\]
证明如下:
\[
|R_\bs{X}(\tau)|=|R_\bs{X}(t,s)|\leq[R_\bs{X}(t,t)\cdot R_\bs{X}(s,s)]^{\frac{1}{2}}= R_\bs{X}(0)
\]
例 1: 调制信号 Modulated Signal
假设有一调制信号:
\[
\bs{X}(t)=\bs{A}(t)\cdot\cos(2\pi ft+\theta)
\]
其中, \(\bs{A}(t)\) 和 \(\theta\) 都是随机且独立的, 并且 \(A(t)\) 满足 W.S.S., \(\theta\sim U(0,2\pi)\). 接下来证明, \(\bs{X}(t)\) 也满足 W.S.S.:
首先, 计算期望:
\[
\begin{aligned}
E(\bs{X}(t))=&E(\bs{A}(t)\cdot\cos(2\pi ft+\theta))\\
=&E(\bs{A}(t))\cdot E(\cos(2\pi ft+\theta))=0\\
\end{aligned}
\]
其次, 计算相关函数:
\[
\begin{aligned}
R_\bs{X}(t,s)=&E(\bs{X}(t)\bs{X}(s))\\
=& E(\bs{A}(t)\cdot\cos(2\pi ft+\theta)\cdot \bs{A}(s)\cdot\cos(2\pi fs+\theta))\\
=& E(\bs{A}(t)\bs{A}(s))\cdot E(\cos(2\pi ft+\theta)\cos(2\pi fs+\theta))\\
=& R_\bs{A}(t-s)\cdot\int_{0}^{2\pi}\frac{1}{2\pi}\cos(2\pi ft+\theta)\cos(2\pi fs+\theta)\rmd\theta\\
=&R_\bs{A}(t-s)\cdot\frac{1}{4\pi}\int_{0}^{2\pi}[\cos(2\pi f(t+s)+2\theta)+\cos(2\pi f(t-s))]\rmd\theta\\
=&R_\bs{A}(t-s)\cdot\frac{1}{4\pi}\left[0+\int_{0}^{2\pi}\cos(2\pi f(t-s))\rmd\theta\right]\\
=&\frac{1}{2}R_\bs{A}(t-s)\cos(2\pi f(t-s))
\end{aligned}
\]
例 2: 随机电报信号/噪声 Random Telegraph Signal/Noise
随机电报信号 (Random Telegraph Signal, RTS)，或随机电报噪声 (Random Telegraph Noise, RTN)，是一种在各种电子器件，特别是微电子器件中常见的低频噪声现象。它的主要特点是器件的电流或电压会在两个或多个离散的电平之间随机跳变.
假设 \(X(t)\) 只在 \(1\) 和\(-1\) 之间跳变, 且从 \(0\) 到 \(s\) 时刻, 总共发生了 \(N(s)\) 次跳变, 其中:
\[
\begin{aligned}
\bs{N}(s)\sim& \ P(\lambda s)\\
\bs{X}(0)\sim& \begin{pmatrix}
1 & -1 \\
\frac{1}{2} & \frac{1}{2}
\end{pmatrix}
\end{aligned}
\]
首先计算期望:
\[
E(\bs{X}(t))=1\cdot P(\bs{X}(t)=1)+(-1)\cdot P(\bs{X}(t)=-1)
\]
用全概率拆分一下:
\[
\begin{aligned}
P(\bs{X}(t)=1)=&P(\bs{N}(t)\ \text{is even}|\bs{X}(0)=1)P(\bs{X}(0)=1)\\
&+P(\bs{N}(t)\ \text{is odd}|\bs{X}(0)=-1)P(\bs{X}(0)=-1)
\end{aligned}
\]
其中, \(\bs{X}(0)\) 的分布已知:
\[
P(\bs{X}(0)=1)=P(\bs{X}(0)=-1)=\frac{1}{2}
\]
而 \(X(t)\) 从 \(0\) 到 \(t\) 发生奇数和偶数次跳变的概率分别为:
\[
\begin{aligned}
P(\bs{N}(t)\ \text{is odd})&=\sum_{k\ \text{is odd}}\frac{(\lambda t)^{k}}{k!}\exp{(-\lambda t)}\\
P(\bs{N}(t)\ \text{is even})&=\sum_{k\ \text{is even}}\frac{(\lambda t)^{k}}{k!}\exp{(-\lambda t)}
\end{aligned}
\]
熟知:
\[
\begin{aligned}
\exp(\lambda t)&=\sum_{k=0}^{+\infty}\frac{(\lambda t)^k}{k!}\\
\exp(-\lambda t)&=\sum_{k=0}^{+\infty}(-1)^k\frac{(\lambda t)^k}{k!}
\end{aligned}
\]
通过加减抵消, 就有:
\[
\begin{aligned}
P(\bs{N}(t)\ \text{is odd})&=\frac{1}{2}[\exp(\lambda t)-\exp(-\lambda t)]\exp(-\lambda t)\\
P(\bs{N}(t)\ \text{is even})&=\frac{1}{2}[\exp(\lambda t)+\exp(-\lambda t)]\exp(-\lambda t)
\end{aligned}
\]
于是:
\[
\begin{aligned}
&&P(\bs{X}(t)=1)=\frac{1}{2}\\
\Rightarrow&&P(\bs{X}(t)=-1)=\frac{1}{2}\\
\end{aligned}
\]
从而期望为零
\[
E(\bs{X}(t))=0
\]
接下来计算相关函数:
\[
\begin{aligned}
R_\bs{X}(t,s)=E(\bs{X}(t)\bs{X}(s))
\end{aligned}
\]
\(R_\bs{X}(t,s)\) 只有两种取值, \(1\) 和 \(-1\), 取决于 \(\bs{X}(t)\) 和 \(\bs{X}(s)\) 是否异号, 这等价于从 \(t\) 到 \(s\) (不妨设 \(t\geq s\)) 过程中发生跳变次数 \(\bs{N}(t-s)\) 的奇偶性. 因此有:
\[
\begin{aligned}
R_\bs{X}(t,s)=&1\cdot P(\bs{N}(t-s)\ \text{is even})+(-1)\cdot P(\bs{N}(t-s)\ \text{is odd})\\
=&\exp(-2\lambda(t-s))
\end{aligned}
\]
严格来讲,
\[
R_\bs{X}(t,s)=\exp(-2\lambda|t-s|)
\]
因此, RTS 是宽平稳的.
深入理解相关函数: Local Determines Global
Theorme: 设一随机过程满足 W.S.S., 那么就有:
\[
R_\bs{X}(0)=R_\bs{X}(T)\ \Leftrightarrow\ R_\bs{X}(T+\tau)=R_\bs{X}(\tau),\ \forall\tau
\]
即, 只要找到某个时刻 \(T\), 在 \(T\) 上的相关与零点一致, 那么相关函数一定是以 \(T\) 为周期的周期函数.
必要性证明是显然的, 充分性需要绕一下.
Lemma: 若 \(R_\bs{X}(0)=R_\bs{X}(T)\), 则:
\[
E|\bs{X}(t)-\bs{X}(t+T)|^2=0
\]
Proof:
\[
\begin{aligned}
E|\bs{X}(t)-\bs{X}(t+T)|^2=&E(\bs{X}^2(t))+E(\bs{X}^2(t+T))-2E(\bs{X}(t)\bs{X}(t+T))\\
=&\ R_\bs{X}(0)+R_\bs{X}(0)-2R_\bs{X}(T)\\
=&\ 0
\end{aligned}
\]
因此,
\[
\begin{aligned}
|R_\bs{X}(\tau)-R_\bs{X}(\tau+T)|
=&\ |E(\bs{X}(0)\bs{X}(\tau))-E(\bs{X}(0)\bs{X}(\tau+T))|\\
=&\ |E(\bs{X}(0)\cdot(\bs{X}(\tau)-\bs{X}(\tau+T))\\
\leq&\ E(|\bsX(0)|\cdot|\bs{X}(\tau)-\bs{X}(\tau+T))|)\\
\leq&\ [E(\bsX(0))^2\cdot E(\bs{X}(\tau)-\bs{X}(\tau+T))^2]^{\frac{1}{2}}\\
=&\ 0
\end{aligned}
\]
从而原命题得证.
由此结论, 一个矩形框一定不会是相关函数. 毕竟有无穷多个点与 \(R(0)\) 取值相同, 周期连续地趋近于 \(0\), 因此这个矩形框非得无限延长为一条直线不可.
Theorem: 如果有一随机过程 \(\bsX(t)\) 是 W.S.S., 且 \(R_\bsX(\tau)\) 在零点连续, 那么 \(R_\bsX(\tau)\) 在任一点连续.
用这一结论也可以判断矩形框不会是相关函数, 明显有间断点嘛.