\[
\newcommand{\bs}{\boldsymbol}
\newcommand{\bsX}{\boldsymbol{X}}
\newcommand{\bf}{\mathbf}
\newcommand{\msc}{\mathscr}
\newcommand{\mca}{\mathcal}
\newcommand{\T}{\text{T}}
\newcommand{\rme}{\mathrm{e}}
\newcommand{\rmi}{\mathrm{i}}
\newcommand{\rmj}{\mathrm{j}}
\newcommand{\rmd}{\mathrm{d}}
\newcommand{\rmm}{\mathrm{m}}
\newcommand{\rmb}{\mathrm{b}}
\newcommand{\and}{\land}
\newcommand{\or}{\lor}
\newcommand{\exist}{\exists}
\newcommand{\sube}{\subseteq}
\newcommand{\lr}[3]{\left#1 #2 \right#3}
\newcommand{\intfy}{\int_{-\infty}^{+\infty}}
\newcommand{\sumfy}[1]{\sum_{#1=-\infty}^{+\infty}}
\newcommand{\vt}{\vartheta}
\newcommand{\ve}{\varepsilon}
\newcommand{\vp}{\varphi}
\newcommand{\Var}{\text{Var}}
\newcommand{\Cov}{\text{Cov}}
\newcommand{\edef}{\xlongequal{def}}
\newcommand{\prob}{\text{P}}
\newcommand{\Exp}{\text{E}}
\newcommand{\t}[1]{\text#1}
\newcommand{\N}{\mathbb{N}}
\newcommand{\Z}{\mathbb{Z}}
\newcommand{\Q}{\mathbb{Q}}
\newcommand{\R}{\mathbb{R}}
\newcommand{\C}{\mathbb{C}}
\newcommand{\versionofnewcommand}{\text{260125}}
\]
\(\mathrm{II.\ 2.}\) Basic Probabiltiy Theory
14. Probability Triple
Definition:
By probability triple, we mean a measure space \((\Omega,\msc{F},\prob)\) of total mass \(\prob(\Omega)=1\). An element \(E\) of \(\msc{F}\) will be called an event, and \(\prob(E)\) will be called the probability of the event \(E\). A statement \(S\) about outcomes \(\omega\) in \(\Omega\) is said to be true almost surely (a.s.) if
\[
F = \{ \omega \ |\ S(\omega)\text{ is true} \} \in \msc{F}\quad \text{and}\quad \prob(F)=1
\]
15. First Borel-Cantelli Lemma
Definition:
Suppose that \(\{E_n\}\) is a sequence of events. We define
\[
\begin{aligned}
\lim\sup E_n =\ & \bigcap_{m} \bigcup_{n \geq m} E_n\\
= &\ \{ \omega \ |\ \text{for every } m,\
\exist \ n(\omega)\geq m \text{ such that }\omega \in E_{n(\omega)} \}\\
= &\ \{ \omega \ |\ \omega\in E_n \text{ for infinitely many n} \}
\end{aligned}
\]
Lemma (Reverse Fatou Lemma for Sets):
\[
\prob(\lim\sup E_n) \geq \lim\sup \prob(E_n)
\]
Lemma (First Borel-Cantelli Lemma):
Let \(\{E_n\}\) be a sequence of events such that \(\sum_n \prob(E_n)<\infty\). Then,
\[
\prob(\lim\sup E_n) = \prob(E_n, \text{i.o.}) = 0
\]
16. Random Variable; Distribution Function
Definition:
Let \((E,\msc{E})\) be a measurable space. By an \((E,\msc{E})\)-valued random variable \(X\) carried by our probability triple \((\Omega,\msc{F},\prob)\), we mean an \((\msc{F/E})\)-measurable map \(X\) from \(\Omega\) to \(E\), so that \(X^{-1}:\msc{E} \to \msc{F}\). By the law \(\Lambda_X\) of \(X\), we mean the probability measure \(\Lambda_X = P \circ X^{-1}\) on \((E,\msc{E})\) so that
\[
\Lambda_X(A) = \prob(X \in A) = \prob(\omega \ |\ X(\omega)\in A)\quad(A\in\msc{E})
\]
Suppose that for \(i=1,2\), \((E_i,\msc{E}_i)\) is a measurable space and that \(X_i\) is an \((E_1,\msc{E}_i)\)-valued random variable. Let
\[
E = E_1 \times E_2,\quad \msc{E}=\msc{E}_1 \times \msc{E}_2
\]
and
\[
X(\omega) = (X_1(\omega),X_2(\omega)) \in E
\]
We say that the joint law \(\Lambda_{X_1,X_2}\) of \(X_1\) and \(X_2\) is then defined to be the law of \(X\).
If the random variable \(X\) is \(\R\)-valued, then it follows from the Uniqueness Lemma and the fact that \(\msc{B}=\sigma(\pi(\R))\), where
\[
\pi(\R)\edef \{(-\infty,x]:x\in\R\},
\]
that the law of \(X\) is determined by the distribution function \(F_X\) of \(X\):
\[
F_X(x) = \prob(X \in (-\infty,x])=\prob(X \leq x),\quad x\in\R
\]
Inversely we can say that the distribution function is a special law of random variables.
17. Expectation