46.1.3. Linear system without exogenous inputs - infinite time horizon

In the infinite-time horizon, under the assumption of stabilizability and observability (todo Prove it!), the optimal control law for a LTI system is a proportional control law,

\[\mathbf{u} = - \mathbf{K} \mathbf{x} = - \mathbf{R}^{-1} ( \mathbf{B}^T \mathbf{P} + \mathbf{S}^T ) \mathbf{x} \ ,\]

with the constant matrix \(\mathbf{P}\) satisfying the algebraic Riccation equation (ARE)

(46.8)\[\mathbf{0} = \mathbf{P} \hat{\mathbf{A}} + \hat{\mathbf{A}}^T \mathbf{P} + \hat{\mathbf{Q}} - \mathbf{P} \mathbf{B} \mathbf{R}^{-1} \mathbf{B}^T \mathbf{P} \ ,\]

with \(\hat{\mathbf{A}} = \mathbf{A} - \mathbf{B} \mathbf{R}^{-1} \mathbf{S}^T\) and \(\hat{\mathbf{Q}} = \mathbf{Q} - \mathbf{S} \mathbf{R}^{-1} \mathbf{S}^T\).

Introducing the relation \(\boldsymbol\lambda(t) = \mathbf{P} \mathbf{x}(t)\) - with constant \(\mathbf{P}\) - into the Hamiltonian system

\[\begin{split} \begin{bmatrix} \dot{\mathbf{x}} \\ \dot{\boldsymbol\lambda} \end{bmatrix} \begin{bmatrix} \mathbf{A} - \mathbf{B} \widetilde{\mathbf{R}}^{-1} \mathbf{S}^T & - \mathbf{B} \widetilde{\mathbf{R}}^{-1} \mathbf{B}^T \\ -\widetilde{\mathbf{Q}} + \mathbf{S} \widetilde{\mathbf{R}}^{-1} \mathbf{S}^T & - \mathbf{A}^T + \mathbf{S} \widetilde{\mathbf{R}}^{-1} \mathbf{B}^T \end{bmatrix} \begin{bmatrix} \mathbf{x} \\ \boldsymbol\lambda \end{bmatrix} \ , \end{split}\]

it’s possible to get 2 “decoupled” (the coupling lies in the relation \(\boldsymbol\lambda = \mathbf{P} \mathbf{x}\)) equations for \(\mathbf{x}\) and \(\boldsymbol\lambda\),

\[\begin{split}\begin{aligned} \dot{\mathbf{x}} & = \left( \hat{\mathbf{A}} - \mathbf{B} \widetilde{\mathbf{R}}^{-1} \mathbf{B}^T \mathbf{P} \right) \mathbf{x} \\ \dot{\boldsymbol{\lambda}} & = - \left( \hat{\mathbf{A}}^T + \hat{\mathbf{Q}} \mathbf{P}^{-1} \right) \boldsymbol\lambda \ . \end{aligned}\end{split}\]

If the matrix \(\mathbf{P}\) is chosen as the solution of the ARE (46.8), then the two equations have opposite eigenvalues, as

\[\begin{split}\begin{aligned} \hat{\mathbf{A}}^T + \hat{\mathbf{Q}} \mathbf{P}^{-1} & = \left( \hat{\mathbf{A}}^T \mathbf{P} + \hat{\mathbf{Q}} \right) \mathbf{P}^{-1} = \\ & = \left( - \mathbf{P} \hat{\mathbf{A}} + \mathbf{P} \mathbf{B} \mathbf{R}^{-1} \mathbf{B}^T \mathbf{P} \right) \mathbf{P}^{-1} = \\ & = - \mathbf{P} \left( \hat{\mathbf{A}} - \mathbf{B} \mathbf{R}^{-1} \mathbf{B}^T \mathbf{P} \right) \mathbf{P}^{-1} \ , \end{aligned}\end{split}\]

the two matrices are related by a similarity transformation, through \(\mathbf{P}\), with a minus sign.

Thus, the transition matrix of the state and the co-state in the closed-loop systems are dual, in the sense of

\[\boldsymbol\Phi_{cl,x}(t,t_0) = \boldsymbol\Phi_{cl,\lambda}^T(t_0,t) \ .\]

Lyapunov criterium for stability. Let \(V(x) = \mathbf{x}^T \mathbf{P} \mathbf{x}\), its time derivative reads

\[\begin{split}\begin{aligned} \dot{V}(x(t)) & = \dot{\mathbf{x}}^T \mathbf{P} \mathbf{x} + \mathbf{x}^T \mathbf{P} \dot{\mathbf{x}} = \\ & = \mathbf{x}^T ( \hat{\mathbf{A}} - \mathbf{B} \mathbf{R}^{-1} \mathbf{B}^T \mathbf{P} )^T \mathbf{P} \mathbf{x} + \mathbf{x}^T \mathbf{P} ( \hat{\mathbf{A}} - \mathbf{B} \mathbf{R}^{-1} \mathbf{B}^T \mathbf{P} ) \mathbf{x} = \\ & = \mathbf{x}^T ( \hat{\mathbf{A}}^T \mathbf{P} + \mathbf{P} \hat{\mathbf{A}} - 2 \mathbf{P} \mathbf{B} \mathbf{R}^{-1} \mathbf{B}^T \mathbf{P} ) \mathbf{x} = \\ & = - \mathbf{x}^T ( \hat{\mathbf{Q}} + \mathbf{P} \mathbf{B} \mathbf{R}^{-1} \mathbf{B}^T \mathbf{P} ) \mathbf{x} = \\ & \le 0 \ , \end{aligned}\end{split}\]

for all \(\mathbf{x} \ne \mathbf{0}\), if the matrix \(\hat{\mathbf{Q}} + \mathbf{P} \mathbf{B} \mathbf{R}^{-1} \mathbf{B}^T \mathbf{P}\) is a positive definite matrix.

Asymptotic stability and detectability of \(\ (\hat{\mathbf{A}}, \hat{\mathbf{Q}}^{1/2})\)

For asymptotic stability, we require the pair \((\hat{A}, \hat{Q}^{1/2})\) to be detectable (see observability and detectability. Here is the proof logic:

• define the null space: suppose there exists a trajectory \(x(t)\) such that \(\dot{V}(x(t)) = 0\) for all \(t \geq 0\).

• analyze the terms: Since both \(\hat{Q}\) and \(PBR^{-1}B^TP\) are positive semi-definite, \(\dot{V} = 0\) implies that \(x^T \hat{Q} x = 0\) and \(x^T PBR^{-1}B^TP x = 0\) simultaneously.

• vanishing control: If \(x^T PBR^{-1}B^TP x = 0\), then the optimal control input \(u = -R^{-1}B^T P x\) must be zero, as

  \[0 = \mathbf{x}^T \mathbf{P} \mathbf{B} \mathbf{R}^{-1} \mathbf{R} \mathbf{R}^{-1} \mathbf{B}^T \mathbf{P} \mathbf{x} = \mathbf{u}^T \mathbf{R} \mathbf{u} \ ,\]

  forces \(\mathbf{u} = \mathbf{0}\), as the weight matrix on the control must be positive definite, \(\mathbf{R} > 0\) (update the condition for mixed weights, and/or weights on performance outputs with \(\widetilde{\mathbf{R}}\)).

• the residual dynamics: in this “unobservable” region where \(\dot{V}=0\), the dynamics simplify to \(\dot{x} = \hat{A} x\).

• the constraint: for \(\dot{V}\) to remain zero, we also require \(\mathbf{x}^T \hat{\mathbf{Q}} \mathbf{x} = 0\), and with \(\hat{\mathbf{Q}}^{1/2}\) as the Choleski factor of the semi-definite positive matrix, \(\hat{\mathbf{Q}}^{1/2} \mathbf{x}(t) = 0\) for all \(t\).

All the statements above can be then summarized as the condition of detectability of a linear system

\[\begin{split}\left\{\begin{aligned} \dot{\mathbf{x}} & = \hat{\mathbf{A}} \mathbf{x} \\ \mathbf{y} & = \hat{\mathbf{Q}}^{1/2} \mathbf{x} \\ \end{aligned}\right.\end{split}\]

todo check the following sentence

If this system is detectable, then any state \(x\) that produces zero output (\(y=0\)) must satisfy \(\lim_{t \to \infty} x(t) = 0\). Since \(y=0\) is a requirement for \(\dot{V}=0\), we have proved that all trajectories must eventually vanish, achieving Global Asymptotic Stability.

Schur decoposition.

\[\begin{split}\mathbf{U}^T \mathbf{H} \mathbf{U} = \mathbf{T} = \begin{bmatrix} \mathbf{T}_{11} & \mathbf{T}_{12} \\ \mathbf{0} & \mathbf{T}_{22} \end{bmatrix}\end{split}\]