46.8. Optimal control for reference tracking

In order to get perfect asymptotic tracking of polynomial reference signals with degree \(m\) with optimal control, the original system must be augmented with the dynamical equations of \(m+1\) integral errors. This process is shown for tracking step reference with the augmentation of the original system with a single integrator and for tracking ramp reference with the augmentation with a double integrator.

This procedure is somehow equivalent to the addition of integrators in open-loop transfer function, as discussed in the section about reference tracking as a performance of a closed-loop system.

An example about optimal control for reference tracking in a mass-damper-spring system is shown in the next section, as a Jupyter notebook that can be opened, run, and modified in Colab.

46.8.1. Augmented system for tracking step reference signal - integrator

Let \(\mathbf{y}_{\text{ref}}\) a reference signal. An augmented system can be defined in order to use optimal control for tracking the desired reference signal.

\[\begin{split}\begin{aligned} \dot{\mathbf{x}} & = \mathbf{A} \mathbf{x} + \mathbf{B} \mathbf{u} \\ \mathbf{y} & = \mathbf{C} \mathbf{x} + \mathbf{D} \mathbf{u} \end{aligned}\end{split}\]

the state-space representation of the plant. Let \(\mathbf{y}_{ref}\) a desired output and the integral error

\[\mathbf{e}(t) := \int_{\tau=0}^{t} \left\{ \mathbf{y}(\tau) - \mathbf{y}_{ref}(\tau) \right\} d \tau \ ,\]

as a new state with dynamical equation

\[\dot{\mathbf{e}} = \mathbf{y}(t) - \mathbf{y}_{ref}(t) = \mathbf{C} \mathbf{x} + \mathbf{D} \mathbf{u} - \mathbf{y}_{ref} \ .\]

The optimal control is applied to the augmented system

\[\begin{split}\underbrace{\begin{bmatrix} \dot{\mathbf{x}} \\ \dot{\mathbf{e}} \end{bmatrix}}_{\dot{\mathbf{z}}} = \underbrace{\begin{bmatrix} \mathbf{A} & \cdot \\ \mathbf{C} & \cdot \end{bmatrix}}_{\hat{\mathbf{A}}} \underbrace{\begin{bmatrix} \mathbf{x} \\ \mathbf{e} \end{bmatrix}}_{\mathbf{z}} + \underbrace{\begin{bmatrix} \mathbf{B} \\ \mathbf{D} \end{bmatrix}}_{\hat{\mathbf{B}}_u} \mathbf{u} + \underbrace{\begin{bmatrix} \cdot \\ -\mathbf{I} \end{bmatrix}}_{\hat{\mathbf{B}}_{ref}} \mathbf{y}_{ref}\end{split}\]

Optimal control framework provides the opitmal gain matrix \(\hat{\mathbf{K}}\), so that \(\mathbf{u} = - \hat{\mathbf{K}} \mathbf{z}\) and the closed loop system becomes

\[\dot{\mathbf{z}} = \left( \hat{\mathbf{A}} - \hat{\mathbf{B}}_u \hat{\mathbf{K}} \right) \mathbf{z} + \hat{\mathbf{B}}_{ref} \mathbf{y}_{ref} \ .\]

\(\lim_{t \rightarrow +\infty} e(t)\)

Transfer function. From the reference signal to the tracking error,

\[\mathbf{e}(s) = \mathbf{y}_{\text{ref}} - \mathbf{G}_k(s) \frac{\mathbf{e}}{s}\]

\[\mathbf{e}(s) = \left[ \mathbf{I} + \frac{1}{s} \mathbf{G}_k(s) \right]^{-1} \mathbf{y}_{\text{ref}}(s)\]

Final value theorem.

\[\lim_{t \rightarrow +\infty} \mathbf{e}(t) = \lim_{s \rightarrow 0} s \mathbf{e}(s)\]

Let \(\mathbf{y}_{\text{ref}}\) a polynomial reference input, \(\mathbf{y}_{\text{ref}} = \mathbf{C}_{-1} \delta(t) + \sum_{k=0}^{m} \mathbf{C}_k t^k\) then its Laplace transform reads \(\mathbf{y}_{\text{ref}}(s) = \sum_{k=-1}^{m} \frac{\mathbf{C}_k}{s^{k+1}}\)

\[\begin{split}\begin{aligned} \lim_{t \rightarrow +\infty} \mathbf{e}(t) & = \lim_{s \rightarrow 0} s \mathbf{e}(s) = \\ & = \lim_{s \rightarrow 0} s \left[ \mathbf{I} + \frac{1}{s} \mathbf{G}_k(s) \right]^{-1} \left( \sum_{k=-1}^{m} \frac{\mathbf{C}_k}{s^{k+1}} \right) = \\ & = \lim_{s \rightarrow 0} s \cdot s \, \mathbf{G}_k^{-1}(s) \left( \sum_{k=-1}^{m} \frac{\mathbf{C}_k}{s^{k+1}} \right) \ , \end{aligned}\end{split}\]

and thus, if

• \(m = 0\) (i.e. linear combination of Dirac’s delta and a step) the asymptotic value of the tracking error is zero;

• \(m = 1\) (i.e. up to a ramp) the asymptotic value of the tracking error is constant, \(\mathbf{G}_k^{-1}(0) \mathbf{C}_1\);

• \(m > 1\) the tracking error goes to infinity.

Fig. 46.2 Caption

46.8.2. Augmented system for tracking ramp reference signal - double integrator

Let \(\mathbf{y}_{\text{ref}}\) a reference signal. An augmented system can be defined in order to used optimal control. Let

\[\begin{split}\begin{aligned} \dot{\mathbf{x}} & = \mathbf{A} \mathbf{x} + \mathbf{B} \mathbf{u} \\ \mathbf{y} & = \mathbf{C} \mathbf{x} + \mathbf{D} \mathbf{u} \end{aligned}\end{split}\]

the state-space representation of the plant. Let \(\mathbf{y}_{ref}\) a desired output and the integral error

\[\mathbf{e}_1(t) := \int_{\tau=0}^{t} \left\{ \mathbf{y}(\tau) - \mathbf{y}_{ref}(\tau) \right\} d \tau \ ,\]

as a new state with dynamical equation

\[\dot{\mathbf{e}}_1 = \mathbf{y}(t) - \mathbf{y}_{ref}(t) = \mathbf{C} \mathbf{x} + \mathbf{D} \mathbf{u} - \mathbf{y}_{ref} \ .\]

The second augmenting state is defined as

\[\mathbf{e}_2(t) := \int_{\tau=0}^{t} \mathbf{e}_1(\tau) \, d\tau \ ,\]

so that the dynamical equation is

\[\dot{\mathbf{e}}_2 = \mathbf{e}_1 \ .\]

The optimal control is applied to the augmented system

\[\begin{split}\underbrace{\begin{bmatrix} \dot{\mathbf{x}} \\ \dot{\mathbf{e}}_1 \\ \dot{\mathbf{e}}_2 \end{bmatrix}}_{\dot{\mathbf{z}}} = \underbrace{\begin{bmatrix} \mathbf{A} & \cdot & \cdot \\ \mathbf{C} & \cdot & \cdot \\ \cdot & \mathbf{I} & \cdot \end{bmatrix}}_{\hat{\mathbf{A}}} \underbrace{\begin{bmatrix} \mathbf{x} \\ \mathbf{e}_1 \\ \mathbf{e}_2 \end{bmatrix}}_{\mathbf{z}} + \underbrace{\begin{bmatrix} \mathbf{B} \\ \mathbf{D} \\ \cdot \end{bmatrix}}_{\hat{\mathbf{B}}_u} \mathbf{u} + \underbrace{\begin{bmatrix} \cdot \\ -\mathbf{I} \\ \cdot \end{bmatrix}}_{\hat{\mathbf{B}}_{ref}} \mathbf{y}_{ref}\end{split}\]

Optimal control framework provides the opitmal gain matrix \(\hat{\mathbf{K}}\), so that \(\mathbf{u} = - \hat{\mathbf{K}} \mathbf{z}\) and the closed loop system becomes

…

\(\lim_{t \rightarrow +\infty} e(t)\)

Transfer function. The closed-loop system reads

\[\begin{split}\begin{aligned} s \mathbf{x} & = \mathbf{A} \mathbf{x} + \mathbf{B} \mathbf{u} = \\ & = \left( \mathbf{A} - \mathbf{B} \mathbf{K} \right) \mathbf{x} - \mathbf{B} \mathbf{K}_1 \mathbf{z}_1 - \mathbf{B} \mathbf{K}_2 \mathbf{z}_2 \\ \mathbf{y} & = \left( \mathbf{C} - \mathbf{D} \mathbf{K}_u \right) \mathbf{x} - \mathbf{D} \mathbf{K}_1 \mathbf{z}_1 - \mathbf{D} \mathbf{K}_2 \mathbf{z}_2 \ . \end{aligned}\end{split}\]

so that

\[\begin{split}\begin{aligned} \mathbf{x} & = \left( s \mathbf{I} - \mathbf{A} + \mathbf{B} \mathbf{K} \right)^{-1} \left[ - \mathbf{B} \mathbf{K}_1 \mathbf{z}_1 - \mathbf{B} \mathbf{K}_2 \mathbf{z}_2 \right] = \\ & = \dots \end{aligned}\end{split}\]

and \(\mathbf{y} = \mathbf{G}_1(s) \mathbf{K}_1 \mathbf{z}_1 + \mathbf{G}_2(s) \mathbf{K}_2 \mathbf{z}_2\).

From the reference signal to the tracking error,

\[\begin{split}\begin{aligned} \mathbf{e}(s) & = \mathbf{y}_{\text{ref}} - \mathbf{y} = \\ & = \mathbf{y}_{\text{ref}} - \mathbf{G}(s) \mathbf{K}_1 \mathbf{z}_1 - \mathbf{G}(s) \mathbf{K}_2 \mathbf{z}_2 = \\ & = \mathbf{y}_{\text{ref}} - \mathbf{G}_1(s) \frac{\mathbf{e}}{s} - \mathbf{G}_2(s) \frac{\mathbf{e}}{s^2} \ , \end{aligned}\end{split}\]

\[\mathbf{e}(s) = \left[ \mathbf{I} + \frac{1}{s} \mathbf{G}_1(s) + \frac{1}{s^2} \mathbf{G}_2(s) \right]^{-1} \mathbf{y}_{\text{ref}}(s)\]

Final value theorem.

\[\lim_{t \rightarrow +\infty} \mathbf{e}(t) = \lim_{s \rightarrow 0} s \mathbf{e}(s)\]

Let \(\mathbf{y}_{\text{ref}}\) a polynomial reference input, \(\mathbf{y}_{\text{ref}} = \mathbf{C}_{-1} \delta(t) + \sum_{k=0}^{m} \mathbf{C}_k t^k\) then its Laplace transform reads \(\mathbf{y}_{\text{ref}}(s) = \sum_{k=-1}^{m} \frac{\mathbf{C}_k}{s^{k+1}}\)

\[\begin{split}\begin{aligned} \lim_{t \rightarrow +\infty} \mathbf{e}(t) & = \lim_{s \rightarrow 0} s \mathbf{e}(s) = \\ & = \lim_{s \rightarrow 0} s \left[ \mathbf{I} + \frac{1}{s} \mathbf{G}_1(s) + \frac{1}{s^2} \mathbf{G}_2(s) \right]^{-1} \left( \sum_{k=-1}^{m} \frac{\mathbf{C}_k}{s^{k+1}} \right) = \\ & = \lim_{s \rightarrow 0} s \cdot s^2 \, \mathbf{G}_2^{-1}(s) \left( \sum_{k=-1}^{m} \frac{\mathbf{C}_k}{s^{k+1}} \right) \ , \end{aligned}\end{split}\]

and thus, if

• \(m = 1\) (i.e. linear combination of Dirac’s delta, step, and ramp) the asymptotic value of the tracking error is zero;

• \(m = 2\) (i.e. up to a parabolic input, \(\dots + \mathbf{C}_2 t^2\)) the asymptotic value of the tracking error is constant, \(\mathbf{G}_2^{-1}(0) \mathbf{C}_2\);

• \(m > 2\) the tracking error goes to infinity.

Fig. 46.3 Caption