Data-driven discovery of Koopman eigenfunctions for control

Data-driven transformations that reformulate nonlinear systems in a linear framework have the potential to enable the prediction, estimation, and control of strongly nonlinear dynamics using linear systems theory. The Koopman operator has emerged as a principled linear embedding of nonlinear dynamics, and its eigenfunctions establish intrinsic coordinates along which the dynamics behave linearly. In this work, we demonstrate a data-driven control architecture, termed Koopman Reduced Order Nonlinear Identification and Control (KRONIC), that utilizes Koopman eigenfunctions to manipulate nonlinear systems using linear systems theory. We approximate these eigenfunctions with data-driven regression and power series expansions, based on the partial differential equation governing the infinitesimal generator of the Koopman operator. Although previous regression-based methods may identify spurious dynamics, we show that lightly damped eigenfunctions may be faithfully extracted using sparse regression. These lightly damped eigenfunctions are particularly relevant for control, as they correspond to nearly conserved quantities that are associated with persistent dynamics, such as the Hamiltonian. We derive the form of control in these intrinsic eigenfunction coordinates and design nonlinear controllers using standard linear control theory. KRONIC is then demonstrated on a number of relevant examples, including 1) a nonlinear system with a known linear embedding, 2) a variety of Hamiltonian systems, and 3) a high-dimensional double-gyre model for ocean mixing.


Introduction
In contrast to linear systems, a generally applicable and scalable framework for the control of nonlinear systems remains an engineering grand challenge. Improved nonlinear control has the potential to transform our ability to interact with and manipulate complex systems across broad scientific, technological, and industrial domains. From turbulence control to brain-machine interfaces, emerging technologies are characterized by high-dimensional, strongly nonlinear, and multiscale phenomena that lack simple models suitable for control design. This lack of simple equations motivates data-driven control methodologies, which include system identification for model discovery [37,4,48,6,10]. Alternatively, one can seek transformations that embed nonlinear dynamics in a global linear representation, as in the Koopman framework [32,42]. The goal of the Koopman control developed in this paper is to reformulate nonlinear dynamics in a linear framework to enable the use of powerful linear optimal and robust control techniques [59,19,62]. The result is an innovative, data-driven mathematical framework for nonlinear control via our KRONIC architecture.
A wide range of data-driven and nonlinear control approaches exist in the literature, including model-free adaptive control [34], extremum-seeking [3], gain scheduling [55], feedback linearization [15], describing functions [68], sliding mode control [20], singular perturbation [31], geometric control [7], back-stepping [30], model predictive control [14,41], reinforcement learning [66], and machine learn-⋆ This paper was not presented at any IFAC meeting. Corresponding author E. Kaiser ing control [22,9]. Although considerable progress has been made in the control of nonlinear systems [34,24,56], methods are generally tailored to a specific class of problems, require considerable mathematical and computational resources, or don't readily generalize to new applications. Currently there is no overarching framework for nonlinear control as exists for linear systems [19,59]. Fortunately, the rise of big data, advances in machine learning, and new approaches in dynamical systems are changing how we approach these canonically challenging nonlinear control problems.
Koopman operator theory has recently emerged as a leading framework to obtain linear representations of nonlinear dynamical systems from data [42]. This operatortheoretic perspective complements the more standard geometric [21] and probabilistic [17,18] perspectives. The ability to embed nonlinear dynamics in a linear framework is particularly promising for the prediction, estimation, and control of nonlinear systems.
In 1931, Koopman showed that a nonlinear dynamical system may be represented by an infinite-dimensional linear operator acting on the space of measurement functions of the state of the system [32]. Formulating dynamics in terms of measurements is appealing in the era of big data. Since the seminal work of Mezić and Banaszuk [46] and Mezić [42], Koopman theory has been the focus of efforts to characterize nonlinear systems. Many classical results have been extended to the Koopman formalism [13,43]. For example, level sets of Koopman eigenfunctions form invariant partitions [12] and may be used to analyze mixing. The Hartman-Grobman theorem has also been generalized to provide a linearizing transform in the entire basin of attraction of a stable or unstable equilibrium or periodic orbit [36].

Incorporate control
Unknown dynamics

Control design
Nonlinear feedback Reduced-order dynamicṡ control [72,11,33,61,1] of nonlinear systems. The Koopman operator is infinite-dimensional, and control laws are typically based on a finite-dimensional approximation. Dynamic mode decomposition (DMD) [57,54,67,35] approximates the Koopman operator with a best-fit linear model. However, DMD is based on linear measurements, which do not span a Koopman invariant subspace for many nonlinear systems [70,11,35]. Current data-driven methods to approximate the Koopman operator include extended DMD (EDMD) [70,71,29] and the variational approach of conformation dynamics (VAC) [49,50]. EDMD was recently used for model predictive control with promising results [33]. However, EDMD models may suffer from closure issues for systems with multiple fixed points or attractors, as a linear model only has a single fixed point [11]. This may lead to spurious or corrupted dynamics [11,35]. For chaotic systems, delay coordinates provides a promising embedding [65,8,2].
Obtaining useful data-driven coordinate transformations that approximate Koopman eigenfunctions remains an open challenge in data-driven dynamical systems [35,11].
In the present work, we develop a strategy to identify lightly damped Koopman eigenfunctions from data and use these eigenfunctions for control. We show that Koopman eigenfunctions provide a principled linear embedding of nonlinear dynamics, resulting in an intrinsic coordinate system to design controllers that manipulate coherent structures using linear control theory. Importantly, we show that even if VAC/EDMD models contain spurious dynamics, the lightly damped eigenfunctions are often not corrupted. For example, the Hamiltonian energy is a Koopman eigenfunction, and we are able to manipulate this function with linear control. These nonlinear control techniques generalize to any lightly damped eigenfunction. As a more sophisticated example, we consider the double gyre flow, which is a model for ocean mixing. The discovery of intrinsic coordinates for optimized nonlinear control establishes our datadriven KRONIC framework 1 , shown in Fig. 1.

Identifying Koopman eigenfunctions from data
The classical geometric theory of dynamical systems considers a set of coupled ordinary differential equations in terms of the state of the system x ∈ M, where M is a differentiable manifold, often given by M = R n . In discrete time, the dynamics are given by x k+1 = F(x k ), where F may be the flow map of the dynamics in (1). In 1931, B. O. Koopman introduced the operator theoretic perspective, showing that there exists an infinitedimensional linear operator, given by K t , that acts to advance all measurement functions g of the state with the flow 1 Code at https://github.com/eurika-kaiser/KRONIC. of the dynamics, K t g = g • F. Thus, the Koopman operator advances measurements linearly: g(x k+1 ) = K t g(x k ). For smooth dynamics, there is a continuous system where K is the infinitesimal generator of the Koopman operator. A Koopman eigenfunction ϕ(x) corresponding to eigenvalue λ satisfies λϕ(x) = ϕ(F(x)). In continuous-time, a Koopman eigenfunction ϕ(x) satisfies Combined with (3), this results in a linear partial differential equation (PDE) for the eigenfunction ϕ(x): This formulation assumes that the eigenfunctions are smooth [45]. It is possible to approximate eigenfunctions with this PDE by solving for the Laurent series [26] or by regression. This assumes continuous and differentiable dynamics.
2.1 Data-driven discovery of continuous-time eigenfunctions Sparse identification of nonlinear dynamics (SINDy) [10] is used to identify Koopman eigenfunctions for a particular value of λ. This formalism assumes that the system has a point or mixed spectrum, for which eigenfunctions with distinct eigenvalues exist. A schematic is displayed in Fig. 2.
First, we build a library of candidate functions: We choose Θ large enough so that the Koopman eigenfunc-

Unknown System
Data Sparse regressioṅ Example:

Sparse coefficients
Example: tion may be well approximated in this library: Given data X = [x 1 x 2 · · · x m ], the time derivativeẊ = [ẋ 1ẋ2 · · ·ẋ m ] can be approximated numerically from x(t) if not measured directly [10]. The total variation derivative [16] is recommended for noise-corrupted measurements. It is then possible to build a data matrix Θ(X): Moreover, we can define a library of directional derivatives, representing the possible terms in ∇ϕ(x) · f (x) from (5): It is then possible to construct Γ from data: For a specific eigenvalue λ, the Koopman PDE in (5) may be evaluated on data, yielding: The formulation in (10) is implicit, so that ξ will be in the null-space of λΘ(X) − Γ(X,Ẋ). The right null-space of (10) for a given λ is spanned by the right singular vectors of λΘ(X) − Γ(X,Ẋ) = UΣV * (i.e., columns of V) corresponding to zero-valued singular values. It may be possible to identify the few active terms in an eigenfunction by finding the sparsest vector in the null-space [53], as in the implicit-SINDy algorithm [38]. In this formulation, the eigenvalues λ are not known a priori, and must be learned online along with the approximate eigenfunction. Koopman eigenfuntions and eigenvalues can also be determined as the solution to the eigenvalue problem Kξ α = λ α ξ α , where K = Θ † Γ is obtained via leastsquares (LS) regression. While many eigenfunctions are spurious, i.e. these eigenfunctions do not behave linearly, those corresponding to lightly damped eigenvalues can be well approximated [11,35]. In either case, it is critical to test any candidate eigenfunctions to make sure that a candidate eigenfunction ϕ(x(t)) actually behaves linearly as the predicted e λt on trajectories x(t).
From a practical standpoint, data in X does not need to be sampled from full trajectories, but can be obtained using more sophisticated strategies such as latin hypercube sampling or sampling from a distribution over the phase space. Moreover, reproducing kernel Hilbert spaces (RKHS) can be employed to describe ϕ(x) locally in patches of M. It may also be possible to directly identify a recursion relationship to obtain a power series expansion as in [26,40].

Data-driven discovery of discrete-time eigenfunctions
In discrete-time, an eigenfunction evaluated at a number of data points in X will satisfy: Again, searching for such an eigenfunction ϕ(x) in a library Θ(x) yields the matrix system: where X ′ = x 2 x 3 · · · x m+1 is a time-shifted matrix. This formalism directly identifies the functional representation of a Koopman eigenfunction with eigenvalue λ.
If we seek the best least-squares fit to (12), this reduces to the extended DMD [70] formulation: Again, it is necessary to confirm that predicted eigenfunctions actually behave linearly on trajectories.

Koopman operator control in eigenfunctions
We now propose a general, data-driven control architecture in Koopman eigenfunction coordinates, referred to as Koopman Reduced Order Nonlinear Identification and Control (KRONIC), to enable the use of powerful optimal and robust control techniques available for linear systems [59,19,62]. First, dominant eigenfunctions are discovered in an unsupervised, data-driven manner (see Sec. 2) providing a reduced-order representation of the system. A feedback controller is then developed in these coordinates, yielding a possibly nonlinear control law in the original state variables. Control in eigenfunction coordinates is quite general, encompassing the stabilization of fixed points and periodic orbits, e.g. via the Hamiltonian eigenfunction, or the manipulation of more general spatial-temporal coherent structures given by level sets of other eigenfunctions.

Control-affine systems
We first examine how adding control to a dynamical system (1) affects a single Koopman eigenfunction. This formulation then readily generalizes to multiple eigenfunctions. Consider a control-affine system d dt with a multi-channel input u ∈ R q and continuously dif- ϕ(x) is a Koopman eigenfunction associated with the unforced dynamics. The control input enters the dynamics of ϕ via an additional term leading to a control-affine system, which is linear in ϕ and possibly nonlinear in the control.

Nonaffine control systems
More generally, a nonlinear system of the following form is considered: We may define a Koopman eigenfunction ϕ(x, u) so that Applying the chain rule, we find that the dynamics of the Koopman eigenfunction depends onu, which is arbitrary: we may specify that ϕ(x, u) reduces to the eigenfunction ϕ(x,ū) ofẋ = f (x,ū) for all lockedū ∈ N , as in [52]. In this case, the eigenfunction is parameterized by the inputū If we augment the eigenfunction vector with the input u, we obtain where we viewu as the input to the Koopman linear system, and the ∇ u ϕ(x, u) matrix varies based on x and u. Thus, we may enact a gain-scheduled control law.

Formulation of the optimal control problem
We now formulate the infinite-horizon, optimal control problem [62] for a reduced set of Koopman eigenfunctions. The control objective is a quadratic cost functional: (20) where ϕ = [ϕ λ1 ϕ λ2 . . . ϕ λr ] T comprises r eigenfunctions with ϕ λj associated with eigenvalue λ j . For this cost to be equivalent to the cost in the original state x, a modified weight matrix may be considered such that ϕ T Q ϕ ϕ ≈ x T Qx. However, this may only be achieved for special invertible eigenfunctions. More generally, the matrix Q ϕ allows one to weight particular eigenfunction directions.
For a control-affine system (14), the dynamics are with Λ = diag(λ 1 , . . . , λ r ). Depending on the structure of ϕ(x) and B, the actuation vector B ϕ = ∇ x ϕ(x) · B may be a function of x. A state-dependent control term may be interpreted as a spatially distributed actuation. A feedback controller is now sought in the Koopman representation, leading to which is linear in the eigenfunctions, but generally nonlinear in the state x. The optimal gain C ϕ can be determined by solving the associated algebraic Riccati equation leading to a linear quadratic regulator (LQR) formulated in Koopman eigenfunctions. We may also consider reference track- (20). The resulting feedback law is optimal for manipulating Koopman eigenfunctions. The controller (22) aims to minimize the value of J(ϕ, u). However, the controller may be directly applied to (14), for which the control problem may be suboptimal and can be interpreted as solving the state-dependent Riccati equation [51] (SDRE). For the general case, it is possible to augment the state with the control input and include the derivative of the control as new inputû :=u: with q × q identity matrix I q . This may be interpreted as integral control. The cost functional is then given by with some restrictions onR. Modifying the system structure, by moving the nonlinearity in the control term into the state dynamics, improves the tractability of the problem [5]. If a closed linear regression model K is found in the space of observables, LQR applied to this system is equivalent to LQR in a suitably chosen subspace of eigenfunctions. However, the matrix K is usually large, as the state is lifted to a high-dimensional space, and the resulting model rarely closes. Model reduction, such as balanced truncation, may be used for real-time applications. Moreover, generalizations such as LQG balancing [25] yield both a reduction for a possibly unstable system along with the compensator. As elaborated above, finding such a closed linear regression model, K, is extremely challenging in practice; instead lightly damped Koopman eigenfunctions can be robustly identified, providing a reduced-order model. Thus, control of a few dominant controllable Koopman eigenfunctions is realizable.

Example: System with a slow manifold
We now demonstrate optimal control in intrinsic Koopman coordinates for a system with quadratic nonlinearity that gives rise to a slow manifold [67]: d dt This system can be represented as a finite-dimensional, linear system in a special choice of observable functions, making it amenable to optimal control [11]. KRONIC in intrinsic coordinates provides a powerful alternative if the system does not allow for a fully controllable, linear representation.
The system exhibits slow and fast dynamics for |λ| ≪ |µ| and has a single fixed point at the origin. This nonlinear system can be embedded in a higher-dimensional space (y 1 , y 2 , y 3 ) = (x 1 , x 2 , x 2 1 ) where the dynamics are linear: The unforced dynamics form a closed linear system in a Koopman-invariant subspace. However, B y may be a function of state x depending on the specific choice of B. Koopman eigenfunctions of the unforced system, i.e. B ≡ [0 0] T , are ϕ µ = x 1 and ϕ λ = x 2 − bx 2 1 with b = λ λ−2µ with eigenvalues λ and µ, respectively. These eigenfunctions remain invariant under the Koopman operator K and can be interpreted as intrinsic coordinates. Note that ϕ pα := ϕ p α are also Koopman eigenfunctions with eigenvalue pα for p ∈ N (and p ∈ Z for non-vanishing ϕ pα ).
The dynamics of the Koopman eigenfunctions are affected by the additional control term B =[0 0] T according to where the first term represents the uncoupled, linear dynamics of the eigenfunctions and the second term a possibly state-dependent control term ∇ϕ · B.
The controller shall stabilize the unstable fixed point at the origin if either µ or λ are unstable. The control objective is to minimize the quadratic cost function J x = lim t→∞ t 0 x T Qx + R u 2 dτ with Q = [ 1 0 0 1 ] and R = 1, weighing state and control expenditures equally. We can also define a cost function in observable functions with Q y = Q 0 0 0 and in intrinsic coordinates with Q ϕ = Q 0 b 0 b b 2 . Control penalization R is kept the same. Here, Q y and Q ϕ are chosen to yield the same cost in x. Linear optimal control is then directly applied to (27). The controller is linear in y and ϕ and yields a nonlinear controller in the state x.
First, we consider the system with µ = −0.1 and λ = 1 with an unstable x 2 direction. The control vector is B =   [0 1] T , resulting in a constant vector in y or ϕ coordinates, B y = B ϕ = [0 1 0] T . Note that the first direction is uncontrollable, but also stable. We then consider the system where the stable and unstable directions are reversed. Standard LQR results are compared (see Fig. 3) for the linearized dynamics, truncated Koopman system in y, truncated Koopman system in ϕ (KRONIC), as well as with feedback linearization [28] (u F L = λx 2 1 − C F L x) and with numerically solving the nonlinear control problem as a twopoint boundary value problem (TPBV), with performance evaluated in terms of J x (t). Both controllers, in observable functions and intrinsic coordinates, achieve the same performance and outperform all other controllers. The results for the truncated Koopman system in observables correspond to those presented in [11]. There is no difference between those results and the control results of the system in intrinsic coordinates, as the systems are connected via an invertible linear transformation. One advantage of a formulation in intrinsic coordinates will become apparent in the next case, where the stable and unstable directions are reversed.
We now consider (25) where the stable and unstable directions are reversed, i.e. µ = 0.1 and λ = −1. The control input now affects the first state x 1 with B = [1 0] T , otherwise this state is uncontrollable. As elaborated in [11], in this case the linear system in observables (26) will become nonlinear in the control term and, more importantly, will become unstabilizeable as the third state y 3 has a positive eigenvalue 2µ. Analogously, the Koopman system in intrinsic coordinates has an uncontrollable, unstable direction in ϕ 2µ . However, the dynamics of the Koopman eigenfunctions are uncoupled, thus the third direction can be discarded and the controller is developed in the controllable subspace: Note that the third direction φ 2µ (x) is a harmonic of φ µ (x), i.e., φ 2µ = φ 2 µ . Thus, these two directions may not be inde-pendently controllable with a single input. Here, the controller for x is determined by solving the SDRE to account for the nonlinear control term. In a truncated Koopman eigenfunction system, the weights in J ϕ can generally not be modified to directly replicate the cost J x . Here, the weights for J x are Q = eye(2) and R = 1, and for J ϕ := J(ϕ, u) are Q ϕ = Q and R ϕ = 4R. The choice for R ϕ ensures a fair comparison by enforcing the same applied energy input for all methods. Performance results are summarized in Tab. 1. It is also possible to combine KRONIC with MPC allowing for more general objective functions. Then, the control could be formulated in terms of J x by computing the inverse ϕ −1 : R r → R n if it exists or estimating x from ϕ using, e.g., multidimensional scaling [27]. Control in Koopman intrinsic coordinates allows one to discard uncontrollable, unstable directions, for which standard control toolboxes such as Matlab's lqr fail. Note that feedback linearization fails in this case too: The control law is of the form u FL = x −1 1 C FL x. As the system approaches the origin, the control input becomes unboundedly large.

Example: Hamiltonian energy control
Conserved quantities, such as the Hamiltonian, are Koopman eigenfunctions associated with the eigenvalue λ = 0. Hamiltonian systems represent a special class of systems for which we can easily discover a Koopman eigenfunction directly from data.
The dynamics of a Hamiltonian system are governed by where q and p are the generalized state and momenta vectors, respectively. The Hamiltonian H = H(q, p) considered here is time-independent, representing the conserved energy in the system. Trajectories of the system evolve on constant energy hypersurfaces {(q, p) : H(q, p) = E}, which may be interpreted as oscillatory modes. Thus, energy level stabilization is a form of oscillation control and corresponds to stabilizing invariant manifolds in phase space. Also nonlinear fixed point stabilization may correspond to stabilizing a particular value of the Hamiltonian energy. Consider the nonlinear, control-affine Hamiltonian system d dt x = f (x) + Bu where f = [∂H/∂p − ∂H/∂q] T , with state vector x = [q p] T ∈ R n , multi-channel input vector u ∈ R q , and constant control matrix B ∈ R n×q . We may develop the control directly for the eigenfunction equation where ϕ = H. This equation also represents the energy conservation law of the system: A change in the energy, i.e. the Hamiltonian, corresponds to the external supplied work via u. The infinite-horizon cost function to be minimized is J = lim t→∞ 1 2 t 0 Q (H(x(t))) 2 + u T (t)Ru(t) dt with scalar Q penalizing energy deviations and R penalizing the cost expenditure. Assuming a single input u, the control law is given by u = −sign(B H ) Q/RH(x) feeding back the current energy level H(x). The ratio Q/R determines how aggressive the controller is. A more aggressive controller, with Q > R, leads to a faster but also more costly convergence to Note that this feedback control law is linear in the Hamiltonian function, but nonlinear in the state x. In the following, we demonstrate the control approach for several Hamiltonian systems by solving the SDRE; an overview is provided in Fig. 4, where colored curves represent Koopman controlled trajectories. We assume Q = R = 1 for all examples.

Frictionless pendulum
A frictionless pendulum given by a negative cosine potential V (x) = − cos(x 1 ) is steered towards different energy levels H(x) = E ∀x, E ∈ {−1, 0, 1, 2}, with feedback law u = −C(H(x) − E). Minimizing the Hamiltonian drives the system to the equilibrium point at the origin with E = −1 (green curve in Fig. 4). Koopman control has improved performance over an LQR controller based on linearized dynamics near the center.

Duffing system
Certain damped or forced oscillators are described by the Duffing equation (Fig. 4). The system is steered towards the energy level E = 0, which corresponds to the separatrix cycle (yellow dashed lines) yielding a periodic solution. The origin is a saddle point leading to a homoclinic orbit defined by x * 2 = ±x * It is possible to stabilize the center fixed points by commanding a lower reference energy; however, because of symmetry in the system, both fixed points are indistinguishable in eigenfunction coordinates. This illustrates a fundamental uncertainty associated with Koopman eigenfunction control.
The error of the regression problem, the computational time for identifying the eigenfunction, and the control  (8) and (9) corresponds to a time instant of the trajectory. The identified Koopman eigenfunction with λ = 0 from 1792 measurements (kink in Fig. 5 . This eigenfunction represents a perfect recovery of the Hamiltonian up to a scaling, as a Hamiltonian multiplied by a constant scalar is also a conserved quantity. Using a larger time step of ∆t = 0.05, 56 measurements are sufficient to learn the eigenfunction with error O(10 −6 ).
6 Example: Basin hopping in a double well A Koopman eigenfunction represents a topography over the state; e.g., the Hamiltonian function depicts the energy landscape of the system. Trajectory control of a set of particles based on a single Koopman eigenfunction is driven by the difference between the current and desired value in this topography. While the Koopman eigenfunction is a global, linear representation of the system, the control of the particle is local, e.g. by solving a suboptimal SDRE. This is illustrated for a particle in an asymmetric double potential Fig. 6(a)). The Hamiltonian is H = x 2 2 /2+V (x 1 ) and the dynamics are d dt For an initial condition in the left well (blue dots in Fig. 6(a)) the controller will fail to steer the state to the fixed point x * = [1 0] T in the center of the right well as the trajectory will become trapped in the bottom of the left well. Instead, the controller must first increase the energy level to the saddle transition, and after the trajectory passes to the right basin, the energy can be decreased further. We propose a switching control strategy that exploits the Koopman eigenfunctions to transport particles between basins of different heights associated with different fixed points. In particular, the following control strategy steers particles from the left to the right basin: H([a, 1])and x 1 ≤ a, −C(x)(H(x) − H([1, 0])) if x 1 > a. H([a, 1]), associated with the saddle point, is first steered onto a trajectory with slightly higher energy than the homoclinic orbit connecting the two basins. On this orbit, control is turned off and the particle travels to the right well exploiting the intrinsic system dynamics. As soon as it passes the saddle point, control is turned on again directing it to the lowest energy level H( [1,0]) at the desired fixed point. The controller is demonstrated for two initial conditions, as shown in Fig. 6(b-d), driving both to the desired energy level.

A particle on an energy level lower than
This controller can be fully derived from data: First, relevant Koopman eigenfunctions can be identified from data, as shown in Sec. 2. By analyzing roots and extrema of the eigenfunction corresponding to λ = 0, equilibrium and saddle points can be identified. The homoclinic and heteroclinic orbits associated with the saddles can be used as natural transit paths between basins. In each basin, eigenfunction control drives the system to the desired state. Future applications for this control strategy include space mission design and multi-stable systems such as proteins.

Example: Double Gyre flow
We now consider a high-dimensional, spatially evolving, non-autonomous dynamical system, with time-dependent Koopman eigenfunctions. The periodically driven double gyre flow models the transport between convection cells in the Rayleigh-Bénard flow due to lateral oscillations, yielding a simple model for the gulf stream ocean front [60]. We employ here the same parameters as in Shadden et al.'s seminal work on Lagrangian coherent structures [58].
In the non-integrable case, with ε > 0, the total derivative of the stream function is given by where the vanishing term in (33) is not displayed. The first term in (34) arises from the time derivative ∂ ∂t Ψ(x, y, t) and is reformulated into a linear-like structure in Ψ: ∂Ψ/∂t = Aπ cos(πf (x, t)) sin(πy)(∂f /∂t) = π tan −1 (πf (x, t))(∂f /∂t)Ψ = A Ψ Ψ. The second term is the time-dependent analogue of the corresponding term in (33). For both cases, ε = 0 and ε > 0, a controller is developed for the stream function. The control is then applied to an ensemble of drifters to steer them towards the level set Ψ = 0.2. As in the previous examples a quadratic cost function with Q = 1 and R = ( 1 0 0 1 ) is considered. In both cases, B Ψ and A Ψ depend on the state, and for ε > 0 also on time. Thus, the state-dependent Riccati equation is solved at each point in space and time. The controller successfully drives an ensemble of drifters distributed over the domain to the desired level, as shown in Fig. 7. Trajectories are integrated using a 4th-order Runge-Kutta scheme from t ∈ [0, 10]. Example trajectories of the non-autonomous system, with and without control, are presented in Fig. 8. Note that in the non-autonomous case, the reference isocurve Ψ REF = 0.2 (white dashed in Fig. 8(a)) oscillates from left to right while being periodically compressed and expanded in x-direction. The particles follow the moving isocurve resulting in a small oscillation around the desired value (see Fig. 8(b)).
Koopman eigenfunction control can be interpreted in two ways: (1) applying control to swimmers or particles in an external field such as a fluid flow or magnetic field; or (2) changing the external field in which the swimmers or particles drift. For the autonomous double gyre, the second case (not shown) is optimal for the stream function, as (33)  and can be precomputed. Control drives the amplitude of the stream function at each point to the desired value. In contrast, the control system for a single drifter includes a nonlinear control term, resulting in a suboptimal controller. While the time-dependency of the stream function poses challenges for control design, it is nevertheless promising.

Discussion and conclusions
In summary, we have presented a data-driven framework to identify leading eigenfunctions of the Koopman operator and use these eigenfunctions to control nonlinear systems. We find that lightly damped or undamped eigenfunctions may be accurately approximated from data via regression, as these eigenfunctions correspond to persistent phenomena, such as conserved quantities. Moreover, these are often the structures that we seek to control, since they affect long-time behavior. Next, we extend the Koopman operator formalism to include actuation, and demonstrate how a nonlinear control problem may be converted into a controlaffine linear problem in eigenfunction coordinates. We have demonstrated the efficacy of this new data-driven control architecture on a number of nonlinear systems, including Hamiltonian systems and a challenging high-dimensional ocean mixing model. These results suggest that identifying and controlling Koopman eigenfunctions may enable significant progress towards the ultimate goal of a universal datadriven nonlinear control strategy.
EDMD with control (EDMDc) has recently been combined with model predictive control with promising results [33]. Both KRONIC and EDMDc provide Koopmanbased linear system identification that can be leveraged for model-based control, such as MPC or LQR, as shown in Fig. 9. However, there are a number of key differences: (1) KRONIC directly identifies Koopman eigenfunctions, while EDMDc approximates the Koopman operator restricted to a high-dimensional span of observables. (2) EDMD augments the state vector with nonlinear measurements, increasing the dimension of the system. In contrast, KRONIC yields a reduced-order model in terms of a few Koopman eigenfunctions. (3) Control is incorporated in EDMDc as an approximated affine linear term. KRONIC derives an expression for how eigenfunctions are affected by control through the generator equation. However, this may render the control term nonlinear in the state. (4) The cost function for EDMDc is defined in the state or measurement space, while KRONIC defines the cost in eigenfunctions; note that these cost functions are not always transferable. (5) Finally, KRONIC readily admits more complicated solutions, such as limit cycle stabilization, as these correspond to level sets of the eigenfunctions, which is more challenging to incorporate with EDMDc.
As with previous studies, this work further cements the importance of accurate identification and representation of Koopman eigenfunctions. Future work will continue to develop algorithms to extract approximate eigenfunctions from data, and it is likely that these efforts will benefit from advances in machine learning. In addition, there is a fundamental uncertainty principle in representing Koopman eigenfunctions, as these eigenfunctions may themselves be irrepresentable, as are the long-time flow maps for chaotic systems. Instead of seeking perfect Koopman eigenfunctions, which may not be attainable, it will be important to incorporate uncertainty quantification into the data-driven Koopman framework. Model uncertainties may then be managed with robust control.
The present work also highlights an important choice of perspective when working with Koopman approximations. Generally, Koopman eigenfunctions are global objects, such as the Hamiltonian energy function. Although a global, linear representation of the dynamics is appealing, there is also information that is stripped from these representations. For example, in the case of the Hamiltonian eigenfunction, information about specific fixed points and spatial locations are folded into a single scalar energy. If the Hamiltonian is viewed as a topography over the phase space, then this eigenfunction only carries information about the altitude, and not the location. In contrast, Lan and Mezić [36] show that it is possible to extend the Hartman-Grobman theorem to the entire basin of attraction of certain fixed points and periodic orbits, providing a local linear embedding of the dynamics. Connecting these perspectives will continue to yield interesting and important advances in Koopman theory. In addition, there are known connections between the eigenvalues of geometric structures in phase space and the spectrum of the Koopman operator [47]. This knowledge may guide the accurate identification of Koopman eigenfunctions using sparsity-promoting regression techniques.
Formulating control in terms of Koopman eigenfunctions motivates additional work to understand how controllability and observability in these coordinates relate to properties of the nonlinear system. The degree of observability and controllability will generally vary with different eigenfunctions, so that it may be possible to obtain balanced realizations. Moreover, classic results, such as the PBH test, indicate that multi-channel actuation may be necessary to simultaneously control different eigenfunctions corresponding to the same eigenvalue, such as the Hamiltonian energy and conserved angular momentum. The additional degrees of freedom arising from multi-channel inputs can also be used for eigenstructure assignment to shape Koopman eigenfunctions [23]. Thus, actuation may modify both the shape of coherent structures (i.e., Koopman modes associated with a particular eigenfunction) and their time dynam-ics. It may also be possible to use Koopman linear embeddings to optimize sensor and actuator placement for nonlinear systems.
Finally, as undamped or lightly damped eigenfunctions correspond to conserved or nearly conserved quantities, there are many potential applications of the proposed control strategy. For example, symmetries give rise to other conserved quantities, which will likewise yield new Koopman eigenfunctions. In many physical systems, simultaneously controlling the system energy and angular momentum may be an important goal. Much of the present work was formulated with the problem of space mission design in mind. Energy efficient transport throughout the solar system has long driven advances in theoretical and computational dynamical systems, and may stand to benefit from control based on Koopman eigenfunctions. More generally, there is a broad range of applications that stand to benefit from improved nonlinear control, include self-driving cars, the control of turbulence, suppressing the spread of disease, stabilizing financial markets, human machine interfaces, prosthetics and rehabilitation, and the treatment of neurological disorders, to name only a few.