Week 8: The generalized stability theory#

In week 7, we learn how to analyze the stability in a numerical scheme and GFD problem. However, there are a few unique cases where the unstable environment doesn’t exist but we still see the growth of unstable mode. You might wonder how is that possible? Atmospheric scientists also found this feature around late 1990s, thus they turned their attention to so-called non-modal system. We will get into more details later; but now, let me quickly recap some concepts we learned over the past few weeks which will be the key ingredients in today’s discussion.

  • Lyapunov exponent (von Neumann Method in numerical modeling)

  • Dynamical system

  • initial uncertainty

We know the weather is intrinsically unpredictable due to the imperfect model physics. However, even in the case where we don’t consider those uncertainties (i.e., the governing equation is deterministic and perfect), we still can’t accurately measure the initial states. Lyapunov exponent is an useful metric for quantifying how the small uncertainties at the initial states grows. Referring back to (3) in week 1 (also (138) here), the variables in a certain dynamical system are specified by finite dimensional state vector \(x\) evolving according to the deterministic equations \(f\).

(138)#\[\frac{d \mathbf{y}}{dt}=\mathbf{f}(\mathbf{y}(t))\]

considering that we initialized the model around the point \(y(t)\) and with sufficiently small error \(\delta y= x\), we can somewhat approximate the model with a linear tangent function.

(139)#\[\frac{d \mathbf{x}}{dt}=\mathbf{A(t)}\mathbf{x}\]

where \(A(t)\) is the linearized function of \(f(t)\) or so-called Jacobian matrix (if you still remember finding a Jacobian matrix is equivalent to using a linear tangent function to approximate nonlinear dynamics). The matrix \(A(t)\) is usually time dependent and its realizations do not commute in most cases (i.e., \(\mathbf{A(t_1)}\mathbf{A(t_2)}\neq \mathbf{A(t_2)}\mathbf{A(t_1)}\)). Therefore, the error growth cannot be determined from the analysis of the eigenvalues and eigenvectors of \(\mathbf{A(t)}\) because when we swap the order of \(\mathbf{A(t)}\), we can have a totally different matrix. This is so-called non-modal system. The generalized instability theory is developed to solve the non-model problem and focuses on propagator operator, which is a matrix mapping the initial error \(\mathbf{x(0)}\) to the error at time \(t\).

(140)#\[\mathbf{x(t)} = \Phi (t,0)\mathbf{x(0)}\]

Once \(\mathbf{A(t)}\) is available, \(\Phi (t,0)\) in (140) is readily calculated and can be written as

(141)#\[\Phi (t,0) = \prod _{i=1}^{t} e^{A(i\tau)}\]

where \(A(i\tau) = A(t_i)\). If \(\mathbf{A}\) is autonomous (time independent), then the propagator is simply the matrix exponential i.e.,

(142)#\[\Phi (t,0) = e^{At}\]

The deterministic error growth is bounded by the optimal growth over the interval [0,t]

(143)#\[||\Phi (t,0)|| \equiv \mathrm{max}_{\mathbf{x(t_0)}}\frac{||\mathbf{x(t)}||}{||\mathbf{x(0)}||}\]

The choice here is based on the absolute amplitude. We can also choose an Euclidean norm, i.e., using the growth of total variance/energy to define the growth of signals \(||\mathbf{x(t)}||^2\).

If we choose an Euclidean norm, (143) can be written as,

(144)#\[\mathbf{\Phi (t,0)}^{T}\mathbf{\Phi (t,0)} \equiv \frac{\mathbf{x(t)}^{T}\mathbf{x(t)}}{\mathbf{x(0)}^{T}\mathbf{x(0)}} = \frac{[e^{\mathbf{A}t}\mathbf{x(0)}]^T[e^{\mathbf{A}t}\mathbf{x(0)}]}{\mathbf{x(0)}^{T}\mathbf{x(0)}} \]

where \(\mathbf{\Phi (t,0)}^{T}\mathbf{\Phi (t,0)}\) indicates how much the signals have amplified. For (144), if we can find a matrix, \(v\), which gives us \([e^{\mathbf{A}t}]^{T}[e^{\mathbf{A}t}]v_i = \lambda_iv_i\), where \(\lambda_i\) is a constant, then (144) will reduce to

(145)#\[\mathbf{\Phi (t,0)}^{T}\mathbf{\Phi (t,0)} v_i \equiv \lambda_i v_i\]

So the amplification we are after is an eigenvalue problem for \(\mathbf{\Phi}^{T}\mathbf{\Phi}\) — equivalently, the \(\lambda_i\) are the squared singular values of \(\mathbf{\Phi}\), and the largest one is what (143) calls \(||\mathbf{\Phi}(t,0)||\).

It is tempting to go one step further and write \(\mathbf{\Phi (t,0)}^T\mathbf{\Phi (t,0)} = e^{2\mathbf{A}t}\), which would turn (145) into an ordinary eigenvalue problem for \(\mathbf{A}\): find \(\lambda_{A,i}\), take the real part as the growth rate and the imaginary part as the oscillation, and recover the traditional stability analysis (von Neumann method). That step is legitimate only when \(\mathbf{A}\) is normal, and the whole of generalized stability theory lives in the gap that opens when it is not. The next section makes this precise.

Note

\(\mathbf{\Phi}^{T}\mathbf{\Phi}=\left(e^{\mathbf{A}t}\right)^{T}e^{\mathbf{A}t}=e^{\mathbf{A}^{T}t}e^{\mathbf{A}t}\), and this collapses to \(e^{(\mathbf{A}+\mathbf{A}^{T})t}\) only when \(\mathbf{A}\) and \(\mathbf{A}^{T}\) commute. Matrix exponentials do not obey \(e^{X}e^{Y}=e^{X+Y}\) in general. Keeping singular values and eigenvalues separate is the single most important bookkeeping habit in this chapter.

Examples of generalized stability problem#

Case 0: a 2x2 example we can do by hand#

Before the numerical cases, here is the smallest example that shows the whole effect analytically. Take

(156)#\[\begin{split}\mathbf{A}=\begin{bmatrix} -\epsilon & 1 \\ 0 & -2\epsilon \end{bmatrix},\qquad 0<\epsilon\ll 1\end{split}\]
  • Eigenvalues: \(-\epsilon\) and \(-2\epsilon\). Both negative, so \(\alpha=-\epsilon<0\) and the system is modally stable. Traditional analysis stops here and reports “stable”.

  • Eigenvectors: \(s_1=(1,0)^{T}\) and \(s_2=(1,-\epsilon)^{T}\). As \(\epsilon\rightarrow0\) these become nearly parallel, and the condition number blows up as \(\kappa\approx 2/\epsilon\).

  • Propagator:

(157)#\[\begin{split}e^{\mathbf{A}t}=\begin{bmatrix} e^{-\epsilon t} & \dfrac{e^{-\epsilon t}-e^{-2\epsilon t}}{\epsilon} \\[2ex] 0 & e^{-2\epsilon t} \end{bmatrix}\end{split}\]
  • Peak amplification: the off-diagonal term is the interesting one. It peaks at \(t^{*}=\ln 2/\epsilon\), where it equals \(1/(4\epsilon)\). So the amplitude amplification is \(\approx 1/(4\epsilon)\) and the energy amplification \(G\approx 1/(16\epsilon^{2})\) — arbitrarily large as \(\epsilon\rightarrow0\), although every mode decays.

  • Mechanism: start from \(\mathbf{x}(0)=(0,1)^{T}\). In the eigenbasis this is the difference of two large, nearly cancelling modal components (it has to be — the eigenvectors are nearly parallel, and we are asking for a vector almost perpendicular to both). Because the two components decay at different rates, the cancellation is progressively lost and the perturbation grows temporarily. This is (147) with \(\phi_1-\phi_2\) made concrete.

Notice how badly the eigenvalues mislead here: they predict decay at rate \(\epsilon\), while the actual transient gain is \(O(\epsilon^{-2})\) in energy. The two statements are both correct and describe different time scales.

Case I#

Here we use two examples to demonstrate the finite amplitude of error growth lead by the non-modal dynamics.

(158)#\[\begin{split}e^{At_i} = \begin{bmatrix} 0.9 & 0.3 \\ 0 & 0.9 \end{bmatrix}\end{split}\]

where \(A\) is autonomous. In this case, we can find the diagonal elements are 0.9 indicating that \(A\) is negative and the signal will decay with the increase of forecast lead. However, if we we choose the right initial state, the initial error can still experience significant growth.

In the following figure, we plot the ratio of variance (\(||e^{2At}||\)) between the initial and the final states as a function of time. Blue curve shows the most unstable case and the red curve shows the result when \([1,0]\) is used as the initial state.

#s = np.random.normal(0, 0.1, 1000)
dim      = 100
growth1  = np.zeros((dim,1))
growth2  = np.zeros((dim,1))
a        = np.array([[0.9,0.3],[0.,0.9]])
x_0      = np.array([[1],[0]])
for tau in range(0,dim):
    G_tau    = np.linalg.matrix_power(a, tau)
    x_tau    = np.matmul(G_tau,x_0)
    h        = np.matmul(np.matrix(G_tau).getH(),G_tau)
    w,v      = np.linalg.eig(h)
    growth1[tau,:] = np.max(w)
    growth2[tau,:] = np.matmul(np.transpose(x_tau),x_tau)/np.matmul(np.transpose(x_0),x_0)

plt.figure()
plt.plot(growth1,'b', label='Nonmodal')
plt.plot(growth2,'r', label='Modal')
plt.legend()
../_images/optimal_growth.png

Fig. 21 The error growth in the optimal case (blue) and using [1,0] as initial state (red)#

In FIG20, we can find that both blue and red curves gradually decay to 0 when t is big enough. However, the blue curve shows transient growth when \(t=15\).

Case II#

I the second case, we choose two stable linear operators (159) but we swap them regularly.

(159)#\[\begin{split}A=\begin{bmatrix} 0 & 1 \\ -\omega_{1,2} & 0 \end{bmatrix}\end{split}\]

where \(\omega_1 = 1/2\) and \(\omega_1 = 3\). In the case where there is only one operator, the system trajectory lies on a constant energy surface. However, if we swap the linear operator for every \(T=\frac{\pi}{2\omega_1}\) and \(T=\frac{\pi}{2\omega_2}\), we can find the amplitude grows exponentially.

../_images/error_growth.png

Fig. 22 Red curve shows the case where we swap the linear operators for every quarter period.#

This feature has very important applications. In mid-latitude regions, the environment is usually not baroclinicly unstable. However, we still observe many baroclinic waves. [] (also see Ch5 in [PH06]) provides a very clean explanation. According to week 2, we know that one necessary criteria of baroclinic instability is the change of sign in background PV gradient. This can lead to counter-propagating baroclinic Rossby wave. However, in an environment where PV gradient never change sign, can we still have counter-propagating wave? The answer is yes.

Similar to FIG19, if there are two modes to construct a barolcinic wave. One decays faster and has shorter period than the other. Their linear combination can lead to the transient growth of the wave signals. If the transient growth over different \(t\) can be accumulated, we can see asymptotically unstable waves.

Why this matters beyond the atmosphere#

The same argument was worked out first in engineering fluid dynamics, and the evidence there is unusually clean. The linearized Navier-Stokes operator for a shear flow (the Orr-Sommerfeld/Squire system) is strongly non-normal, and the classical modal predictions fail outright:

  • Plane Couette flow and pipe flow are linearly stable at all Reynolds numbers — \(\alpha<0\) everywhere, so modal analysis says they can never become turbulent.

  • Plane Poiseuille flow becomes linearly unstable only at \(\mathrm{Re}\approx5772\).

Yet all three transition to turbulence at Reynolds numbers far below those thresholds. The resolution is non-modal growth, through the lift-up and Orr mechanisms, which give \(G_{\mathrm{max}}\sim O(\mathrm{Re}^{2})\). That is enough amplification to push a small disturbance into the nonlinear regime, where the linear analysis no longer applies and the flow never gets the chance to follow its modal fate. It is the same story as FIG19 and Case 0, with \(\mathrm{Re}\) playing the role of \(1/\epsilon\).

This is why modern stability analysis reports \(G(t)\), singular value decompositions and pseudospectra alongside the eigenvalues, rather than the eigenvalues alone.

def f(L,y):
    return np.matmul(L,y)

# define RK4 
def rk4(L,t0,y0,tn,n):
    
    # Calculating step size
    h = (tn-t0)/n
    
    #print('\n--------SOLUTION--------')
    #print('-------------------------')    
    #print('x0\ty0\tyn')
    #print('-------------------------')
    for i in range(n):
        k1 = h * (f(L,y0))
        k2 = h * (f(L,(y0+k1/2)))
        k3 = h * (f(L,(y0+k2/2)))
        k4 = h * (f(L,(y0+k3)))
        k = (k1+2*k2+2*k3+k4)/6
        yn = y0 + k
        #print('%.4f\t%.4f\t%.4f'% (x0,y0,yn) )
        #print('-------------------------')
        y0 = yn
        #print(y0)
        t0 = t0+h
    return yn

# see Ch5 of Palmer and Hagedorn 2008 for more details: https://www.cambridge.org/core/books/predictability-of-weather-and-climate/9A8E7E0A16BC8BA928243F46ED192FE6 
# using RK4 for numerical integration
omega1  = 3
omega2  = 0.5
A1      = np.array([[0,1],[-omega1**2,0]])
A2      = np.array([[0,1],[-omega2**2,0]])
v       = np.array([[1],[0]])
record  = np.zeros((1000,2))
record[0,:] = np.reshape(v,[2,])
record2 = np.zeros((10,1000,2))
t       = 0
count   = 0
for i in range(10):
    # swapping linear operator for every quarterly period
    if (i % 4) == 0:
      A=A1
      omega=omega1
    elif (i % 4) == 1:
      A=A2
      omega=omega2
    elif (i % 4) == 2:
      A=A1
      omega=omega1
    elif (i % 4) == 3:
      A=A2
      omega=omega2
    v_template       = v
    for j in range(999): # fixed operator
        t_index          = np.linspace(0,20,1000)
        v_new            = rk4(A,t_index[j],v_template,t_index[j+1],100)
        record2[i,j,:]   = np.reshape(v_new,[2,])
        v_template       = v_new
    for j in range(100): # swapping linear operators 
        t_index         = np.linspace(0,np.pi/(2*omega),101)
        v_new           = rk4(A,t_index[j],v,t_index[j+1],100)
        record[count,:] = np.reshape(v_new,[2,])
        count           = count+1
        v               = v_new


fig=plt.figure()
plt.plot(record2[0,0:110,0],record2[0,0:110,1],'k--')
plt.plot(record2[1,0:700,0],record2[1,0:700,1],'k--')
plt.plot(record2[2,0:110,0],record2[2,0:110,1],'k--')
plt.plot(record2[3,0:650,0],record2[3,0:650,1],'k--')
plt.plot(record[0:400,0],record[0:400,1],'r')
plt.xlim([-40,40])
plt.ylim([-40,40])
plt.xticks(fontsize=20)
plt.yticks(fontsize=20)
fig.set_size_inches(18.5, 10.5)

The application in data assimilation#