State-Space Models and Transfer Functions

Where we are. Part I learned policies and values from interaction, with the dynamics unknown or only sampled. Part II turns to control theory, which starts from the opposite end — an explicit model of the dynamics — and asks what optimal feedback that model permits. The foundational object is the linear state-space model $\dot{\statevec} = \statemat\statevec + \inputmat u$, $y = \outputmat\statevec

• \feedmat u$, and its frequency-domain twin, the **transfer function**$H(s) = \outputmat(s I - \statemat)^-1\inputmat + \feedmat$. They are two views of one system. This chapter sets up that object — and shows its discrete-time form is exactly the known MDP dynamics that LQR (Week 13) will solve in closed form.

The linear state-space model

The running example is the mass–spring–damper $m\ddot{q} + c\dot{q} + kq = u$: taking the state $\statevec = (q, \dot q)$ gives $\statemat = \begin{pmatrix} 0 & 1 \\ -k/m & -c/m \end{pmatrix}$, $\inputmat = \begin{pmatrix} 0 \\ 1/m \end{pmatrix}$, and $\outputmat = \begin{pmatrix} 1 & 0 \end{pmatrix}$ if we measure position. A second-order mechanical law has become a first-order vector ODE — the form every method in Part II acts on.

The transfer function

Laplace-transforming the state equation at zero initial condition and eliminating the state gives the input–output map directly.

For the mass–spring–damper the single-input single-output transfer function is $H(s) = \tfrac{1}{m s^2 + c s + k}$ — the poles are the roots of the characteristic polynomial, i.e. the eigenvalues of $\statemat$, exactly the natural frequencies of the oscillator. Åström and Murray \AAström & Murray (2021) develop both views and their interplay; the state-space form carries the internal dynamics, the transfer function the external behaviour.

Two views, one system

The two representations are equivalent, with one asymmetry worth making precise: the transfer function is unique, but the state-space realization is not — any invertible change of state coordinates leaves the input–output behaviour untouched.

So the state-space model is a coordinate choice on the same dynamics, and the transfer function is the coordinate-free invariant.

Discretization: the bridge to a step map

Control plants are continuous, but computation and the MDP view are discrete. Exact zero-order-hold (ZOH) discretization at step $\stepsize$ — holding $u$ constant across the interval — gives

the matrix exponential of $\statemat\stepsize$. The cheap alternative, forward Euler, takes $\discA \approx I + \statemat\stepsize$ — the first two terms of that exponential, accurate only for small $\stepsize$. Either way the result $\statevec_{k+1} = \discA\statevec_k + \discB u_k$ is a deterministic step map.

The dynamic-programming bridge

That step map closes the loop with Part I. $\statevec_{k+1} = \discA\statevec_k + \discB u_k$ is precisely a Markov decision process’s transition — deterministic, linear, and known — where Chapters 1–10 had a stochastic kernel learned from samples. Control theory’s premise is that the model is in hand, so the Bellman recursion can be solved exactly rather than approximated from experience. Three threads run forward from here:

• To stability and structure (Week 12). Before designing any controller, ask what the model permits: stability from the eigenvalues of $\statemat$, and whether the input can steer and the output can observe the full state.
• To LQR (Week 13). Put a quadratic cost on this linear model and the Bellman equation has a closed-form solution — the Riccati equation — the exact dynamic-programming fixed point Chapter 1’s HJB aside promised.
• To SSMs (Week 24). The very same $(\statemat,\inputmat,\outputmat,\feedmat)$ and its ZOH discretization are a structured-state-space neural layer (S4, Mamba); the control plant and the sequence model share one skeleton.

What’s next

• Week 12 asks what can be known about a state-space model before designing a controller: stability (eigenvalue locations, Lyapunov), and the structural properties of controllability and observability that decide whether control and estimation are even possible.

Exercises

1. (Derive) From $m\ddot{q} + c\dot{q} + kq = u$ with state $\statevec=(q,\dot q)$, derive the matrices $\statemat,\inputmat,\outputmat$ and the transfer function $H(s) = 1/(m s^2 + c s + k)$.

   Solution

   $\dot q_1 = q_2$ and $\dot q_2 = (u - c q_2 - k q_1)/m$ give the stated $\statemat, \inputmat$; measuring position gives $\outputmat = (1\ 0)$. Then $H(s) = \outputmat(sI-\statemat)^{-1}\inputmat = \tfrac{1}{\det(sI-\statemat)}\cdot\tfrac1m = \tfrac{1}{m s^2 + c s + k}$, since $\det(sI-\statemat) = s^2 + (c/m)s + k/m$.

2. (Prove) Show the transfer function is invariant under a state-similarity transform $\statevec = Tz$ (Prop. 11.1), identifying where $T$ cancels.

   Solution

   The transformed system is $(T^{-1}\statemat T, T^{-1}\inputmat, \outputmat T, \feedmat)$; using $sI - T^{-1}\statemat T = T^{-1}(sI-\statemat)T$ and inverting, the $T$ and $T^{-1}$ on either side of $(sI-\statemat)^{-1}$ cancel against $\outputmat T$ and $T^{-1}\inputmat$, returning $H(s)$.

3. (Compute) For $\statemat=\begin{pmatrix}0&1\\-2&-3\end{pmatrix}$, $\inputmat=\begin{pmatrix}0\\1\end{pmatrix}$, $\outputmat=(1\ 0)$, $\feedmat=0$, compute $H(s)$ and its poles.

   Solution

   $\det(sI-\statemat) = s^2+3s+2 = (s+1)(s+2)$, and $H(s) = 1/(s^2+3s+2)$. Poles at $s=-1,-2$ — the eigenvalues of $\statemat$, both in the left half-plane (stable, as Week 12 will formalize).

4. (Implement) In the companion, verify the SS→TF→SS round-trip preserves the transfer function, the mass–spring–damper TF matches $1/(ms^2+cs+k)$, and the step response settles to the DC gain $H(0)=1/k$.

   Solution

   See experiments/python/week11/test_state_space.py: python-control converts SS↔TF (round-trip preserves the rational function up to a state similarity); the mass–spring–damper TF coefficients match the analytic form; and the unit-step final value equals the final-value-theorem prediction $H(0)=1/k$.

5. (Extend) Compare exact ZOH ($\discA = e^{\statemat\stepsize}$) against forward Euler ($\discA \approx I + \statemat\stepsize$): how does the discretization error scale with $\stepsize$?

   Solution

   ZOH is exact for piecewise-constant input; Euler is its first-order truncation, with error $O(\stepsize^2)$ per step (it drops the $\tfrac12\statemat^2\stepsize^2$ and higher terms of the exponential). The companion shows the gap vanishing as $\stepsize\to0$ and growing — eventually destabilizing Euler — for large $\stepsize$.

Companion code

The Week-11 companion lives at experiments/python/week11/ and is the first control companion (Python, on scipy + the python-control library).

• state_space.py — builds the mass–spring–damper state-space model, converts SS↔TF (python-control), simulates impulse/step/forced responses, and discretizes via exact ZOH (scipy.linalg.expm) and forward Euler.
• test_state_space.py — mathematical-correctness tests: the SS→TF→SS round-trip preserves the transfer function; the mass–spring–damper TF equals $1/(ms^2+cs+k)$; the step response settles to the DC gain $H(0)=1/k$; ZOH equals $e^{\statemat\stepsize}$ while Euler is its first-order truncation; and the transfer function is invariant under a random state-similarity transform (Prop. 11.1).

# core algorithms + correctness tests (scipy + python-control)
PYTHONPATH=. pytest experiments/python/week11/test_state_space.py -q

# worked mass-spring-damper SS/TF + response plots (saved locally, not committed)
PYTHONPATH=. python experiments/python/week11/state_space.py --plot