\[ \newcommand{\bs}{\boldsymbol} \newcommand{\bsX}{\boldsymbol{X}} \newcommand{\bf}{\mathbf} \newcommand{\msc}{\mathscr} \newcommand{\mca}{\mathcal} \newcommand{\T}{\text{T}} \newcommand{\rme}{\mathrm{e}} \newcommand{\rmi}{\mathrm{i}} \newcommand{\rmj}{\mathrm{j}} \newcommand{\rmd}{\mathrm{d}} \newcommand{\rmm}{\mathrm{m}} \newcommand{\rmb}{\mathrm{b}} \newcommand{\and}{\land} \newcommand{\or}{\lor} \newcommand{\exist}{\exists} \newcommand{\sube}{\subseteq} \newcommand{\lr}[3]{\left#1 #2 \right#3} \newcommand{\intfy}{\int_{-\infty}^{+\infty}} \newcommand{\sumfy}[1]{\sum_{#1=-\infty}^{+\infty}} \newcommand{\vt}{\vartheta} \newcommand{\ve}{\varepsilon} \newcommand{\vp}{\varphi} \newcommand{\Var}{\text{Var}} \newcommand{\Cov}{\text{Cov}} \newcommand{\edef}{\xlongequal{def}} \newcommand{\prob}{\text{P}} \newcommand{\Exp}{\text{E}} \newcommand{\t}[1]{\text#1} \newcommand{\N}{\mathbb{N}} \newcommand{\Z}{\mathbb{Z}} \newcommand{\Q}{\mathbb{Q}} \newcommand{\R}{\mathbb{R}} \newcommand{\C}{\mathbb{C}} \newcommand{\tb}{\text{b}} \newcommand{\versionofnewcommand}{\text{260125}} \]

1. Bifurcations in Neuron Dynamics

1.1 Basic Concepts of Dynamical System Theory

1.1.1 Dynamical Systems

Consider systems of the ODEs of the form

\[ \dot{X} = F(X), \quad , X = (X_1, \cdots, X_m)^\T \in \R^m \ . \]

Note that we have implicitly assumed that the vector-function $F = (F_1, \cdots ,F_m)^\T : \R^m \to \R^m$ is sufficiently smooth.

Flows and Orbits

A flow of the dynamical system is a function $\Phi_t : \R^m \to \R^m$ parameterized by $t$ such that $X(t) = \Phi_t(X_0)$ is a solution of the system starting from $X_0$, i.e.

\[ \frac{\rmd \Phi_t(X_0)}{\rmd t} \Bigg|_{t=\tau} = F(\Phi_\tau(X_0)) \ ,\quad \Phi_0(X_0) = X_0\ . \]

The flow might not be defined for all $t$ since solutions can escape in a finite time.

Fix $X\in\R^m$, and let $I \subset \R$ be the interval for which $\Phi_t(X)\ ,\ t\in I\ ,$ is defined. Then $\Phi(X) : I \to \R^m$ defines a solution curve, or trajectory, or orbit, of $X$.

Equivalence of Dynamical Systems

Two dynamical systems

\[ \dot{X} = F(X) \ ,\ X \in \R^m\ , \quad \text{and} \quad \dot{Y} = G(Y) \ ,\ Y \in \R^m \]

are said to be topologically equivalent if there is a homeomorphism $h : \R^m \to \R^m$ taking each orbit $\Phi_t(X)\ ,\ X\in\R^m$ of one of the systems to an orbit $\Psi_t(Y)\ , \ Y\in\R^m$ of the other one. Note that the homeomorphism is not required to preserve parameterization by time, that is, the time index can be different. And, if it preserves the time parameterization, then the equivalence is called conjugacy.

Two dynamical systems are said to have qualitatively similar behavior if they are topologically equivalent. A dynamical system is structurally stable if it is topologically equivalent to any $\ve$-perturbation $\dot{X}=F(X)+\ve P(X)$ where $\ve\ll 1$ and $P$ is smooth enough.

When the map $h$ is not a homeomorphism, dynamical system $\dot{Y}=G(Y)$ is said to be a model $\dot{X} = F(X)$. In this case $Y$ is usually belongs to a space smaller than $\R^m$.

Bifurcations

Consider a dynamical system

\[ \dot{X} = F(X,\lambda)\ ,\quad X\in\R^m\ , \]

where $\lambda \in \Lambda$ summarizes system parameters and $\Lambda$ is a Banach space (often Euclidean space $\R^l$). A parameter value $\lambda_0$ is said to be regular, or non-bifurcational if there is an open neighborhood $W$ of $\lambda_0$ such that any system $\dot{X}=F(X,\lambda)$ for $\lambda \in W$ is topologically equivalent to the system $\dot{X}=F(X,\lambda_0)$.

Let $\Lambda_0 \sube \Lambda$ note the set of all non-bifurcation values of $\lambda$. The set $\Lambda_b := \Lambda \setminus \Lambda_0$ is called the bifurcation set. Any $\lambda \in \Lambda_b$ is said to be a bifurcation value.

Equilibria of Dynamical Systems

A point $X_0 \in \R^m$ is an equilibrium if $F(X_0) = 0$. The orbit $\Phi(X_0)$ of the equilibrium consists of one point, the $X_0$. Local behavior at the equilibrium $X_0$ depends on the eigenvalues of the Jacobian matrix

\[ L = D_X F = \lr({\frac{\partial F_i}{\partial X_j}})_{i,j = 1,\cdots,m} \]

evaluated at $X_0$.

When $L$ has no eigenvalues with zero real part, it is said to be hyperbolic. When all eigenvalues have negative real part, it is said to be stable hyperbolic. In both cases $L$ is non-singular ($\det L\neq 0$). Note that local hyperbolicity does not imply global hyperbolicity.

When $L$ has at least one eigenvalue with zero real part, it is said to be non-hyperbolic. $L$ can still be non-singular, which correspond to the case for Andronov-Hopf bifurcations.

Theorem 1.1 (Hartman-Grobman)

A locally hyperbolic dynamical system

\[ \dot{X} = F(X) \]

is locally topologically conjugate to its linearization

\[ \dot{X} = LX \ . \]

We can get a further proposition that locally hyperbolic dynamical systems are locally structurally stable; that is, any $\ve$-perturbation $\dot{X}=F(X)+\ve P(X)$ is locally topologically equivalent to $\dot{X} = F(X)$, which can be assured by the implicit function theorem.

Center Manifold Theory

Non-hyperbolic equilibria often correspond to bifurcations in the system. The analysis of local behavior can be simplified by means of center manifold reduction, through which the dimension along a direction with trivial behavior can be eliminated.

Consider a dynamical system

\[ \dot{X} = F(X)\ , \quad X\in \R^m \]

at a non-hyperbolic equilibrium $X=0$. Let $J$ be the Jacobian matrix at the equilibrium. Suppose that $\{v_1,\cdots,v_{m}\} \sube \R^m$ is the set of generalized eigenvectors of $L$ such that the following hold:

Let the eigenvectors $\{v_1,\cdots,v_{m_1}\}$ correspond to the eigenvalues of $L$ having negative real part. And let

$$ E^{\text{s}} = \text{span}{v_1,\cdots,v_{m_1}} $$

be the eigen-subspace spanned by these vectors. It is called the stable subspace, or the stable linear manifold, for the linear system $\dot{X} = LX$.

Let the eigenvectors $\{v_{m_1+1},\cdots,v_{m_2}\}$ correspond to the eigenvalues of $L$ having positive real part. And let

$$ E^{\text{u}} = \text{span}{v_{m_1+1},\cdots,v_{m_2}} $$

be the eigen-subspace spanned by these vectors. It is called the unstable subspace.

Let the eigenvectors $\{v_{m_2+1},\cdots,v_{m}\}$ correspond to the eigenvalues of $L$ having zero real part. And let

$$ E^{\text{c}} = \text{span}{v_1,\cdots,v_{m_1}} $$

be the corresponding center subspace.

The phase space $\R^m$ is a direct sum of the subspaces; that is,

\[ \R^m = E^{\text{s}} \oplus E^{\text{u}} \oplus E^{\text{c}}\ . \]

The dynamics along $E^{\text{s}}$ and $E^{\text{u}}$ is simple, either exponentially converging or diverging from $X=0$. And we set $E^{\text{h}} = E^\text{s} \oplus E^\text{u}$ where $h$ stands for hyperbolic. The dynamics along $E^{c}$ can be quite complicated, but it can be separated from the other dynamics, as the following theorem says:

Theorem 1.2 (Center Manifold Theorem)

There exists a nonlinear mapping

\[ H : E^{\text{c}} \to E^\text{h}\ ,\quad H(0)=0 \ ,\quad DH(0) = 0 \]

and a neighborhood $U$ of $X=0$ such that the center manifold

\[ M = \{\ x+H(x)\ |\ x \in E^\text{c} \} \]

has the following properties:

(Invariance) The center manifold $M$ is locally invariant with respect to the original dynamical system $\dot{X} = F(X)$. More precisely, if an initial state $X(0)\in M\cap U$, then $X(t)\in M$ as long as $X(t)\in U$. That is, $X$(t) can leave $M$ only when it leaves the neighborhood $U$.
(Attractivity) If the unstable subspace $E^{\text{u}} = \{ 0 \}$, then the center manifold is locally attractive. All solutions staying in $U$ tend exponentially to some solution to the dynamical system on the center manifold $M$.
(Tangency) The center manifold $M$ passes through the origin $X=0$ and is tangent to $E^\text{c}$ at the origin.

Note that the center manifold is not unique.

1.2 Local Bifurcations

Below we use the additive and the Wilson-Cowan models to illustrate some local bifurcations.

\[ \dot{x} = -x + S(\rho+cx) \]

where $x\in\R$ describes the internal parameters of the neuron, $\rho\in\R$ is the external input and the feedback $c\in\R$ characterized the nonlinearity of the system.

1.2.1 Saddle-Node

A one-dimensional dynamical system

\[ \dot{x} = f(x,\lambda)\ , \quad x\in\R\ ,\quad \lambda\in\R^n\ , n>0 \]

is at a saddle-node bifurcation point $x_\text{b}$ for some value $\lambda_\tb$ if the following conditions are satisfied:

Point $(x_\tb,\lambda_\tb)$ is an equilibrium.

$$ f(x_\tb,\lambda_\tb)=0\ . $$

$x_\tb$ is a non-hyperbolic equilibrium.

$$ \frac{\partial f}{\partial x}(x_\tb,\lambda_\tb) = 0\ . $$

The function $f$ has a nonzero quadratic term at the bifurcation point.

$$ \frac{\partial^2 f}{\partial x^2}(x_\tb,\lambda_\tb) \neq 0\ . $$

The function $f$ is non-degenerate with respect to bifurcation parameter $\lambda=(\lambda_1,\cdots,\lambda_n)\in\R^n$.

\[ \frac{\partial f}{\partial \lambda}(x_\tb,\lambda_\tb) = \lr({\frac{\partial f}{\partial \lambda_1},\cdots,\frac{\partial f}{\partial \lambda_n}})^\T\neq0\ . \]

1.2.4 Andronov-Hopf Bifurcations

Andronov-Hopf bifurcations are among the most important bifurcations in neuronal dynamics.

First consider the two-dimensional simple form

\[ \begin{cases} \ \ \dot{x}\quad =\quad f(x,y,\boldsymbol{\lambda})\\ \ \ \dot{y}\quad =\quad g(x,y,\boldsymbol{\lambda}) \end{cases}\ , \quad x,y \in \R,\quad \boldsymbol{\lambda} \in \R^n,\ n>0 \]

near an equilibrium point $(x_b,y_b) \in \R^2$ for parameter vector $\boldsymbol{\lambda}$ near some bifurcation value $\boldsymbol{\lambda}_b$. The dynamical system is said to be an Andronov-Hopf bifurcation if the following conditions are satisfied:

The Jacobian matrix

$$ L=\frac{D(f,g)}{D(x,y)}= \begin{pmatrix} \displaystyle{\frac{\partial f}{\partial x}} & \displaystyle{\frac{\partial f}{\partial y}}\ \displaystyle{\frac{\partial g}{\partial x}} & \displaystyle{\frac{\partial g}{\partial y}} \end{pmatrix} $$

evaluated at the bifurcation point has a pair of purely imaginary eigenvalues, which is equivalent to the conditions:

$$ \text{tr}(L) = \frac{\partial f}{\partial x}+\frac{\partial g}{\partial y}=0 \quad \text{and} \quad \det(L) = \frac{\partial f}{\partial x}\frac{\partial g}{\partial y}-\frac{\partial f}{\partial y}\frac{\partial g}{\partial x}>0. $$

Transversality Condition: Since $\det(L)>0$, the Jacobian matrix is not singular. The implicit function theorem ensures that there is a unique family of equilibria $(x(\boldsymbol{\lambda}),y(\boldsymbol{\lambda})) \in R^2$ for $\boldsymbol{\lambda}$ close to $\boldsymbol{\lambda}_b$. If we let $\alpha(\boldsymbol{\lambda}) \pm \mathrm{i}\omega(\boldsymbol{\lambda})$ denote the eigenvalues of $L(\boldsymbol{\lambda})$ evaluated at the $(x(\boldsymbol{\lambda}),y(\boldsymbol{\lambda})) \in \R^2$, then we say the transversality condition when the $n$-dimensional vector $$ \frac{\mathrm{d}\alpha}{\mathrm{d}\boldsymbol{\lambda}}(\boldsymbol{\lambda})\ =\ \left( \frac{\partial \alpha}{\partial \lambda_1}, \cdots, \frac{\partial \alpha}{\partial \lambda_n} \right) \neq \boldsymbol{0}, \quad \text{where}\ \boldsymbol{\lambda}=(\lambda_1,\cdots,\lambda_n)^\text{T}, $$

Here we consider a typical example of a dynamical system, i.e. the normal form, at Andronov-Hopf bifurcations described by

\[ \begin{cases} \ \ \dot{x} \quad = \quad \alpha x-\omega y + (\sigma x - \gamma y)(x^2 + y^2)\\ \ \ \dot{y} \quad = \quad \omega x+\alpha y + (\gamma x + \sigma y)(x^2 + y^2) \end{cases}\ . \]

The equilibrium point of the system is $(x,y)=(0,0)$. The Jacobian matrix $J$ of the linear part is given as

\[ J=\begin{pmatrix} \alpha & -\omega\\ \omega & \alpha \end{pmatrix} \quad \Rightarrow \quad \lambda = \alpha \pm \mathrm{i}\omega \]

The conditions of Andronov-Hopf bifurcations requires that $\alpha=0$ and $\omega \neq 0$. And parameters $\sigma,\gamma$ in the third-order nonlinear terms controls the properties of limit cycle generated by the bifurcation.

The normal form is often concisely written in a complex coordinate $z=x+\mathrm{i}y$ as

\[ \dot{z} = (\alpha+\mathrm{i}\omega)z+(\sigma+\mathrm{i}\omega)z|z|^2\ , \]

where $|z|^2=x^2+y^2$ describes the distance from the equilibrium point. And $\sigma\neq0$ judges the types/stability of the Andronov-Hopf bifurcation, that is, $\sigma < 0 \Leftrightarrow$ stable limit cycle (supercritical) and $\sigma<0 \Leftrightarrow$ unstable limit cycle (subcritical).

Obviously the form of polar coordinates may also be useful with $z=r\cdot\exp(\mathrm{i}\varphi)$, resulting in

\[ \begin{cases} \begin{aligned} \ \ \dot{r} \quad &= \quad \alpha r + \sigma r^3 \\ \ \ \dot{\varphi} \quad &= \quad \omega + \gamma r^2 \end{aligned} \end{cases}\ . \]

We have introduced a new parameter $\gamma$, which describes how the frequency depends on the amplitude $r$ of oscillators, and is often chosen irrelevant or assumed to be zero in applications. (Since dynamics of the system for $\gamma=0$ and $\gamma\neq0$ are topologically equivalent. )

Normal Forms

The term normal form needs some explanation. Consider a smooth dynamical system

\[ \dot{\bf{X}} = F(\bf{X}),\quad \bf{X} = (X_1,\cdots,X_m) \in \R^m \]

for which $\bf{X}=\bf{0}$ is an equilibrium $\Leftrightarrow\ F(\bf{0})=\bf{0}$. Let $\lambda_1,\cdots,\lambda_m$ be the eigenvalues of the Jacobian matrix $\bf{L}=\bf{D}F$ at $\bf{X}=\bf{0}$, and $\bs{v}_1,\cdots,\bs{v}_m$ be the corresponding eigenvectors. Without loss of generality, we may assume that $\bf{L}$ is given in Jordan normal form. And further assume that the eigenvalues are distinct, so that $\bf{L}$ is diagonal. We say that there is a (non-linear) resonance if there is an integer-valued of the form

\[ \lambda_i = n_1\lambda_1 + \cdots + n_m\lambda_m\quad \text{for some }\lambda_i\quad \text{and} \quad \ n_i\in\N , \quad\sum_{j=1}^m n_j\geq 2\ . \]

For each resonance, we can associate a corresponding resonant monomial

\[ \bs{v}_i X_1^{n_1}\cdots X_m^{n_m} \]

The Poincaré-Dulac Theorem: There exists a near identity change of variables

\[ \bf{X} = \bf{Y} + P(\bf{Y}),\quad P(\bf{Y})=\bf{0},\ \bf{D}P(\bf{0})=\bf{0} \]

that transforms the dynamical system to

\[ \dot{\bf{Y}} = \bf{L}\bf{Y} + W(\bf{Y})\ , \]

where the non-linear vector-function $W$ consists of only resonant monomials. Such a system is called a normal form, through which all the non-resonant monomials are removed. If there is no resonance, then the dynamical system is linearized as $\dot{\bf{Y}}=\bf{L}\bf{Y}$.

Weakly Connected Neural Networks (WCNNs)

We want to study the local dynamics of a WCNN

\[ \begin{cases} \ \ x_i = f_i (x_i,y_i,\boldsymbol{\lambda})+ \varepsilon p_i(x_1,\cdots,x_n,y_1,\cdots,y_n)\\ \ \ y_i = g_i (x_i,y_i,\boldsymbol{\lambda})+ \varepsilon q_i(x_1,\cdots,x_n,y_1,\cdots,y_n) \end{cases} \]

when each neural oscillator is at an Andronov-Hopf bifurcation. Still we often consider the complex form

\[ \dot{z}_i=(\alpha_i+\mathrm{i}\omega_i)z_i + (\sigma_i+\mathrm{i}\gamma_i)z_i|z_i|^2+ \sum_{j=1}^n c_{ij}z_j\ , \]

where $c_{ij}$ are complex-valued coefficients that depend on the matrices

\[ S_{ij}=\frac{\partial(p_i,q_i)}{\partial(x_j,y_j)}= \begin{pmatrix} \displaystyle{\frac{\partial p_i}{\partial x_i}} & \displaystyle{\frac{\partial p_i}{\partial y_i}}\\ \displaystyle{\frac{\partial q_i}{\partial x_i}} & \displaystyle{\frac{\partial q_i}{\partial y_i}} \end{pmatrix} \]