Analysis and design of model predictive control frameworks for dynamic operation An overview ⋆

This article provides an overview of model predictive control (MPC) frameworks for dynamic operation of nonlinear constrained systems. Dynamic operation is often an integral part of the control objective, ranging from tracking of reference signals to the general economic operation of a plant under online changing time-varying operating conditions. We focus on the particular challenges that arise when dealing with such more general control goals and present methods that have emerged in the literature to address these issues. The goal of this article is to present an overview of the state-of-the-art techniques, providing a diverse toolkit to apply and further develop MPC formulations that can handle the challenges intrinsic to dynamic operation. We also critically assess the applicability of the di ﬀ erent research directions, discussing limitations and opportunities for further research.


Introduction
Model predictive control (MPC), also called receding horizon control, is a modern optimization-based control method.The underlying principle is to repeatedly solving finite horizon open-loop optimal control problems online.Feedback is generated implicitly by only implementing the initial part of the optimized input trajectory and repeating the online optimization in the next time step.MPC is widely used in practice (cf. the surveys by Qin and Badgwell (2003) and Samad et al. (2020)) and actively researched in academia (Mayne, 2014).This success of MPC is primarily due to some intrinsic advantages: (i) direct consideration of state and input constraints; (ii) applicability to general nonlinear MIMO systems; (iii) optimization of general performance criteria.
There have been significant advances in academia over the last decades, resulting in a mature stability theory for MPC (Mayne et al., 2000).
Much ongoing research in MPC is focused on deriving efficient implementations (Verschueren et al., 2022), accounting for model errors (Kouvaritakis and Cannon, 2016;Mayne, 2016), or learning the model online (Hewing et al., 2020).In contrast, this article focuses on the design and analysis of MPC framework that can be applied to dynamic operation.

Dynamic operation
The motivation of the present article comes from many emerging control applications in which the control goal is not ⋆ Johannes Köhler was supported by the Swiss National Science Foundation under NCCR Automation (grant agreement 51NF40 180545) accurately reflected by the setpoint stabilization problem, which is classically studied in MPC.We specifically consider the challenges intrinsic in dynamic operation, which has received less attention in the MPC community and is rarely studied in a unified fashion.By dynamic operation, we primarily consider the following three challenges related to the control goal: (C.1) Stationary operation is not desired; (C.2) Desired mode of operation changes online in an unpredictable fashion; (C.3) Desired mode of operation cannot be directly specified in terms of a given setpoint/trajectory of the system state.
First, we illustrate the challenges (C.1)-(C.3)using applications.Consider a motion planning or trajectory optimization problem, as encountered in robotics, aerospace, or autonomous driving.(C.1):The primary goal is to track/follow some dynamic trajectory/path.(C.2):This reference is often generated online by a separate unit (e.g., using artificial intelligence and visual feedback) independent of the controller (and hence unpredictably).(C.3):The reference is primarily specified in terms of a low dimensional output of the system, which is often not physically realizable due to the dynamics or constraints on the system.As a completely different application, consider a heating, ventilation, and air conditioning (HVAC) system to regulate temperature in a building.(C.3):The control goal naturally revolves around economic criteria such as the energy consumption.(C.2):The optimal mode of operation depends on external factors, such as temperature, occupancy, and price/demand, which fluctuate in an unpredictable fashion.(C.1):The same external variables are subject to significant changes over time, e.g., due to the periodic day-night cycle, making the optimal mode of operation non-stationary.Analogous considerations apply to water distribution networks, power networks, power generation using kites and many more.Overall, in many applications the desired mode of operation is dynamic; the controller needs to change the mode of operation based on external, online changing, variables; and the optimal mode of operation is not explicitly specified in terms of a known feasible setpoint/trajectory of the system state.

Contribution
We provide an overview of recent advances in the design and analysis of MPC formulations that can accommodate and address the challenges intrinsic to dynamic operation (C.1)-(C.3).We are specifically interested in classical system theoretic properties, such guaranteeing recursive feasibility, constraint satisfaction, and stability/performance for general nonlinear systems.We provide a set of tools and methods to design MPC schemes for such challenging control applications with guaranteed closed-loop properties.We discuss the existing work and methods in a broad context, highlighting gaps in the existing literature and discussing different strategies in a unified fashion.In general, the challenges (C.1)-(C.3)and the studied methods touch on a number of different research fields in MPC, e.g.: trajectory tracking (Faulwasser, 2012) output regulation (Köhler et al., 2022), artificial references (Limón et al., 2018), economic MPC (Faulwasser et al., 2018), and MPC without terminal constraints (Grüne and Pannek, 2017).We study these formulations in a unified way, focusing on how these frameworks address the challenges outlined above.

Outline
First, we introduce preliminaries regarding the design of a stabilizing MPC scheme and explain the challenges in applying this design to dynamic operation (Sec.2).Moreover, we present designs for the stabilizing terminal ingredients, primarily the local control Lyapunov function (CLF), for general tracking problems (Sec.3).Then, we show how infeasible and online changing references can be accommodated using artificial reference trajectories (Sec.4).Furthermore, we discuss how economic performance criteria can be directly optimized using economic MPC formulations (Sec.5).Finally, we present an alternative framework, focusing on the analysis of simpler MPC schemes without stabilizing terminal ingredients (Sec.6).The paper concludes with a discussion regarding the different provided tools, their benefits, limitations, and open issues in the literature (Sec.7).

Notation
We denote the set of integers in an interval [a, b] by I [a,b] .For k, N ∈ I ≥0 , the modulo operator is denoted by mod(k, N) ∈ I [0,N−1] .The quadratic norm w.r.t. a positive definite matrix Q = Q ⊤ is denoted by x 2 Q := x ⊤ Qx.The identify matrix is denoted by I n ∈ R n×n .For x ∈ R n1 , y ∈ R n 2 , we abbreviate the stacked vector as (x, y) := [x ⊤ , y ⊤ ] ⊤ ∈ R n 1 +n 2 .For a symmetric matrix A = A ⊤ , A ≻ 0 (A 0) indicates that the matrix is positive definite (positive semidefinite).The interior of a set X ⊆ R n is denoted by int(X).By K ∞ , we denote the class of functions α : R ≥0 → R ≥0 , which are continuous, strictly increasing, unbounded, and satisfy α(0) = 0.For a continuously differentiable function F(x, y), F : R n 1 × R n 2 → R m , ∂F ∂x ( x,ȳ) ∈ R m×n 1 denotes the Jacobian matrix of F w.r.t.x evaluated at ( x, ȳ).

Preliminaries: Recursive feasibility & stability in MPC
In the following, we briefly describe the basic MPC algorithm, including its closed-loop properties, analogous to the textbooks by Rawlings et al. (2017) and Grüne and Pannek (2017).We consider a nonlinear discrete-time system with the state x(t) ∈ X = R n , the control input u(t) ∈ U ⊆ R m , and the time step t ∈ I ≥0 .For a sequence u ∈ U N , we denote the k-th element by u k ∈ U with k ∈ I [0,N−1] .Given an initial state x ∈ X and an input sequence u ∈ U N , we denote the solution to (1) after k ∈ I ≥0 steps by x u (k, x) ∈ X, k ∈ I [0,N] with x u (0, x) = x.The system is subject to pointwise-in-time constraints which, e.g., model actuator limitations or safety critical limits on the state.The goal is to minimize a given stage cost ℓ : X × U → R, resulting in the following optimal control problem

Stabilizing MPC design
Solving Problem (3) is typically computationally intractable.MPC approximately solves Problem (3) by repeatedly solving a finite-horizon open-loop optimal control problem: with prediction horizon N ∈ I ≥1 , a terminal cost V f : X → R, and a terminal set X f ⊆ X.Throughout this article, we assume that the constraint set Z is compact, the dynamics f are continuous, and the cost functions ℓ, V f are continuous.Hence, Problem (4) has a minimizer (Rawlings et al., 2017, Prop. 2.4), which is denoted by u ⋆ (x) ∈ U N . 1 The closed-loop operation is defined by i.e., at each time t we apply the first part of the optimal openloop input sequence u ⋆ (x(t)) ∈ U N computed based on the measured state x(t).
The following result recaps standard design conditions and resulting closed-loop properties of the MPC, assuming a feasible steady state (x r , u r ) ∈ Z, f (x r , u r ) = x r should be stabilized.

Assumption 2. (Terminal ingredients)
There exists a terminal control law k f : X f → U such that for all x ∈ X f : (T.1) Constraint satisfaction: (x, k f (x)) ∈ Z; Furthermore, there exists a function α V ∈ K ∞ such that for any state x ∈ X such that Problem (4) is feasible, it holds: (T.4) Weak controllability: J ⋆ N (x) ≤ α V ( x − x r ).Theorem 1. (Rawlings et al., 2017, Thm. 2.19) Suppose Problem (4) is feasible with x = x 0 .Then, Problem (4) is feasible for all t ∈ I ≥0 , the constraints (2) are satisfied, x r is asymptotically stable, and the following performance bound holds for the resulting closed-loop system (5): The intuition behind the terminal set and terminal cost is to approximate the infinite-horizon tail for k ∈ I ≥N (Chen and Allgöwer, 1998), i.e., a feasible solution to Problem (3) is given by appending u ⋆ (x) ∈ U N with the terminal control law k f (x) for k ∈ I ≥N due to (T.1), (T.2).Stability is ensured by showing that the value function J ⋆ N is a Lyapunov function, i.e., Condition (T.3) implies J ⋆ N (x(t + 1)) − J ⋆ N (x(t)) ≤ −ℓ(x(t), u(t)), t ∈ I ≥0 .Condition (T.4) is a technical condition to ensure that J ⋆ N is a Lyapunov function, which holds trivially if x r ∈ int(X f ) (Rawlings et al., 2017, Sec. 2.4.2).Under additional technical conditions, the performance bound (6) also yields a suboptimality/regret bound w.r.t. the optimal performance J ⋆ ∞ (cf., Köhler (2021, App. A) and Grüne and Pannek (2017, Thm. 5.22)).Overall, Theorem 1 provides all the desired closed-loop properties and the posed conditions (Asm.1-2) can be constructively satisfied (cf.Sec. 3).

Challenges in dynamic operation
In the following, we explain how the challenges related to dynamic operation (C.1)-(C.3)complicate the application of this design and how the different frameworks discussed in this article approach these problems.The design of the terminal ingredients (Asm.2) is centred around the CLF V f and a control law k f that locally stabilizes the steady state x r .This offline design becomes challenging if the setpoint x r can change online (C.2) and non-stationary references are considered (C.1).Constructive designs addressing this issue are presented in Section 3.
The desired setpoint x r can change arbitrary online (C.2) and may even be physically infeasible (C.3).This can lead to infeasibility of Problem (4) due to the terminal set constraint X f and hence invalidate all closed-loop guarantees.In Section 4, artificial references are included in the MPC formulation to avoid these complications.
All of these designs try to stabilize some given reference using a positive definite stage cost ℓ (Asm.1).Section 5 shows how to directly minimize an economic (indefinite) cost ℓ (C.3).
Lastly, Section 6 explores an alternative approach: deriving system theoretic conditions and a sufficiently long prediction horizon N to ensure closed-loop properties for simpler MPC designs, which do not require V f , X f satisfying Assumption 2.

Terminal ingredients for nonlinear tracking MPC
In this section, we focus on constructing a terminal cost V f and terminal set X f (Asm.2) for dynamic tracking problems.We first summarize the standard linearization-based design for the regulation problem (Sec.3.1) and discuss extensions to track online changing setpoints (Sec.3.2).Then, we consider tracking of dynamic reference trajectories (Sec.3.3), including the case where the full reference may change online (Sec. 3.4).Lastly, we provide some discussion (Sec.3.5).
For the following exposition, we consider a quadratic stage cost ℓ : assume that the reference r = (x r , u r ) ∈ R n+m lies in some constraint set Z r ⊆ int(Z), and suppose that the dynamics f are twice continuously differentiable.

Regulation problem
We consider the basic stabilizing MPC formulation introduced in Section 2 and provide a constructive design to satisfy Assumptions 1-2.This method was first derived by Chen andAllgöwer (1998), cf. Rawlings et al. (2017, Sec. 2.5.5) for the considered discrete-time variant.We denote the Jacobians by and abbreviate the following matrix expression related to the linear quadratic regulator (LQR): The condition LQR 0 can enforce Condition (T.3) from Assumption 2 for the linearization around the setpoint r ∈ Z r .We account for the linearization error by imposing a stronger condition on the linearization with a tuning variable ǫ > 0. In particular, we choose matrices P, K such that LQR(A(r), B(r), K, P, P, Q + ǫI n , R) 0. This corresponds to designing an LQR for the linearized dynamics around the setpoint r ∈ Z r , which is feasible if (A(r), B(r)) is stabilizable.
Proposition 1.There exists a sufficiently small α > 0, such that the quadratic terminal cost V f (x) = x − x r 2 P , the linear terminal controller k f (x) = u r + K(x − x r ) and the terminal set Twice continuous differentiability of f ensures that Condition (T.3) holds for the nonlinear system for all α ≤ α 1 , with some α 1 > 0. Analytic formulas for α 1 can be derived using bounds on the Hessian (Chen and Allgöwer, 1998;Rawlings et al., 2017) and less conservative constants can be obtained using sampling-based approaches, cf.Chen and Allgöwer (1998, Rk. 3.1), Köhler et al. (2020a, Alg. 1), Griffith et al. (2018); Rajhans et al. (2019).Assumption 1 and Condition (T.4) are trivially satisfied with quadratic bounds.Condition (T.2) follows from Condition (T.3) with ℓ ≥ 0 and the parametrization of the terminal set X f , i.e., sublevel sets of a Lyapunov function are positively invariant.Lastly, Condition (T.1) holds for all α ≤ α 2 with some α 2 > 0 since (x r , u r ) ∈ Z r ⊆ int(Z).In case Z is a polytope, α 2 can be exactly determined using a linear program (Conte et al., 2016, Eq. ( 10)) and Köhler (2021, Lemma 3.37) provide a similar procedure if Z is given by Lipschitz continuous inequality constraints.

Setpoint tracking
The provided design requires some offline computation to determine a terminal cost V f and a terminal set X f to stabilize a specific setpoint x r .This offline procedure needs to be repeated if we wish to stabilize a different setpoint x r , which is undesirable from a practical perspective.Due to its practical relevance, many approaches have been suggested to overcome this issue.Findeisen et al. (2000) suggest a fixed matrix P for the quadratic terminal cost V f for different setpoints x r using a pseudo linearisation, however, practical application of this theory is difficult.Wan andKothare (2003, 2004) locally describe the nonlinear system as a linear difference inclusion (LDI) and compute constant matrices P, K, which are valid for a local set of steady states, i.e., enforcing LQR(A(r), B(r), K, P, P, Q, R) 0 for all r in some region.Limón et al. (2018, App. B) partition the steady-state manifold and compute different matrices P, K for each partition using this LDI description, resulting in a piece-wise quadratic terminal cost.However, the manual partitioning can result in a cumbersome design process, especially if the system is strongly nonlinear or the steady-state manifold is high-dimensional, and the resulting discontinuity can bring additional complications.Köhler et al. (2020a) alleviate these shortcomings with a continuous parametrization: The local CLF condition (T.3) then holds with α(r) chosen sufficiently small (cf.Sec.4.2.5) if LQR(A(r), B(r), K(r), P(r), P(r), Q + ǫI n , R) 0 (9) holds for all feasible reference setpoints r.By interpreting the reference r as a parameter, this can be viewed as a special case of gain-scheduling synthesis for linearparameter varying (LPV) systems, a classical robust control problem (Rugh and Shamma, 2000).The computation of parametrized matrices P, K satisfying Condition (9) can be reformulated as linear matrix inequalities (LMIs, Apkarian and Tuan (2000)), cf.Köhler et al. (2020a) for details.As a result, suitable terminal ingredients can be obtained during runtime by simply evaluating the parametrized terminal ingredients around a new setpoint x r .

Trajectory tracking
In the following, we consider the more general case of tracking a time-varying reference trajectory r(t), t ∈ I ≥0 .We assume that the reference is feasible, i.e., x r (t + 1) = f (x r (t), u r (t)), (x r (t), u r (t)) ∈ Z r , t ∈ I ≥0 .For a given state x and time t, the trajectory tracking MPC is characterized by and value function J ⋆ N (x, t).The closed-loop system is given by The analysis and closed-loop properties of such a trajectory tracking MPC are analogous to Section 2.
Assumption 3.There exists a terminal control law k f : X × I ≥0 → U such that for all t ∈ I ≥0 and all x ∈ X f (t): Furthermore, there exists a function α V ∈ K ∞ , such that for any (x, t) such that Problem (10) is feasible, it holds (T.4) Weak controllability: Problem (10) is feasible for all t ∈ I ≥0 , the constraints (2) are satisfied, and x r (t) is uniformly asymptotically stable for the resulting closed-loop system (11).
The proof and closed-loop properties are analogous to Theorem 1 using J ⋆ N (x, t) as a uniform time-varying Lyapunov function (Grüne and Pannek, 2017, Thm. 2.22).Next, we focus on the constructive design of terminal ingredients for such timevarying reference trajectories r(t) (Asm.3).The following design is largely based on the work by Faulwasser and Findeisen (2011);Faulwasser (2012).Analogous to the stabilization problem, we pick the linear-quadratic time-varying parametrization By linearizing Condition (T.3) and adding a slack ǫ > 0, we obtain the design conditions LQR(A(r(t)), B(r(t)), K(t), P(t), P(t + 1), Q + ǫI, R) 0, (12) which need to hold for all t ∈ I ≥0 .These conditions correspond to an LQR problem for a linear time-varying (LTV) system.Tractable designs can, e.g., be obtained by assuming that the reference r(t) becomes constant after some finite time.In this case, a stationary LQR is solved at the final state and then the time-varying LQR is solved backwards to obtain P(t), K(t) (Faulwasser and Findeisen, 2011).The terminal set scaling α(t) can be chosen similar to the setpoint stabilization problem with the difference that α(t + 1) and α(t) are coupled through the time-varying invariance condition (T.2), cf.Faulwasser and Findeisen (2011, Thm. 2).Aydiner et al. (2016) propose a design for periodic reference trajectories, i.e., r(t) = r(t + T ), ∀t ∈ I ≥0 with some T ∈ I ≥1 .By imposing the same periodicity in P(t), K(t), the design can be cast as a periodic LQR problem or a sequence of T coupled LMIs.

Trajectory tracking with online changing trajectories
The design of a trajectory tracking MPC scheme (Sec.3.3) requires computing the Jacobian of the nonlinear dynamics around the full reference trajectory r(t) and solving a set of LMIs or Riccatti equations to determine the LQR-based terminal ingredients.This can become computationally very expensive.As a result, if the reference trajectory changes during runtime, re-computing terminal ingredients for a new reference r(t) becomes a bottleneck in the practical application.To circumvent this issue, Köhler et al. (2020a) propose a reference generic offline computation.The goal is to have one offline computation to obtain continuously parametrized terminal ingredients of the form P(r), K(r), such that LQR(A(r), B(r), K(r), P(r), P(r + ), Q + ǫI, R) 0 (13) for all r, r + ∈ Z r with x + r = f (x r , u r ).This problem can be cast as a gain-scheduling problem for LPV systems by viewing the reference r as a parameter that is slowly time-varying, where a bound on the change can be deduced from the dynamics.There exists much literature on the synthesis of gain-scheduled controllers, which are used by Köhler et al. (2020a, Prop. 1) to derive a finite-dimensional semidefinite program (SDP) to compute K(r), P(r) satisfying (13).By defining P(t) = P(r(t)), K(t) = K(r(t)), these parametrized terminal ingredients also satisfy the LQR conditions (12) for any possible reference.Hence, the trajectory tracking MPC (Thm.2) can be implemented without a priori knowledge of the full reference and requires no repeated offline computations in case the reference changes.

Discussion
Sections 3.1-3.4provide offline design methods for the terminal ingredients that ensure desirable closed-loop properties for nonlinear tracking MPC schemes.The efficient update of the scaling α > 0 characterizing the terminal set X f and general feasibility and stability questions under online changes of the reference are revisited in Section 4. In the following, we broadly discuss the applicability of these methods and mention variations.

Design complexity
The computation of parametrized terminal ingredients for setpoint tracking (cf.( 9)) is comparatively simple, as the conditions only need to be enforced on the (typically low dimensional) steady-state manifold.On the other hand, the design conditions (13) for dynamic trajectories are enforced over the set of feasible dynamic reference trajectories Z r ⊆ R n+m , which is a significantly higher dimensional set.Hence, direct application for large scale systems is challenging (cf.also Sec. 7.2.2).2However, the benefits of parametrized terminal ingredients are also significantly more pronounced for dynamic reference trajectories as the design of terminal ingredients for a specific reference r(t) (Sec.3.3) can already become cumbersome.

More general stage cost
In case of a positive semidefinite stage cost ℓ, the stability results in Theorem 1/2 are not directly applicable since As-sumption 1 does not hold.However, in case the cost ℓ satisfies a detectability condition, a Lyapunov function can be constructed (Rawlings et al., 2017, Thm. 2.24), compare also Section 6.2.2.Considering the design of the terminal cost V f using LQR: The same procedure can be applied for nonquadratic (twice cont.differentiable) stage costs ℓ by replacing Q + K ⊤ RK by the Hessian of ℓ, see Amrit et al. (2011, Sec. 4.1) and Köhler et al. (2020a, App. D) for details.

LPV & incremental system properties
The design conditions (9)/(13) are comparable to analysis and synthesis conditions for LPV systems.There exists a rich history on addressing nonlinearity and non-convexity using such LPV embeddings in the context of MPC, compare the survey by Morato et al. (2020).Condition (13) ensures that any dynamically feasible reference r can be stabilized, which is also referred to as incremental stability with the contraction metric P. In fact, this design is related to the long history on the interplay between LPV systems, incremental Lyapunov functions, and contraction metrics, compare Angeli (2002); Fromion and Scorletti (2003); Wang et al. (2019); Koelewijn et al. (2019) and Köhler (2021, Appendix C).The presented quadratic terminal cost V f is only a local CLF, however, one can also construct a "globally" valid CLF V f using P(r).5

Tracking MPC formulations using artificial references
In this section, we extend the tracking MPC formulations studied in Section 3 to ensure closed-loop properties under online changing references (C.2), even if the desired target value is not achievable (C.3).Specifically, this is achieved by jointly optimizing artificial references.We first present a setpoint tracking MPC with corresponding closed-loop properties (Sec.4.1) and discuss practical implications (Sec.4.2).Then, we present extensions to periodic reference tracking (Sec.4.3) and discuss how partially decoupling tracking and planning can reduce the computational demand (Sec.4.4).

Setpoint tracking MPC
In the following, we show how artificial references can be used to design a tracking MPC for online changing setpoints.Specifically, we consider an exogenous target value y d (t) ∈ Y = R p for a system output y = h(x, u) ∈ Y, with h Lipschitz continuous.Ideally, the controller drives the system to a steady state that achieves this target output y d .Since this is in general not feasible, we minimize a continuous offset cost V o : Y×Y → R ≥0 that characterizes the difference between the system output y at some steady state and the target output y d .

Setpoint tracking MPC -historical developments
The main difficulty stems from two factors: (i) y d can arbitrarily change online and hence terminal set constraints centred around y d are not recursively feasible; (ii) tracking y d is in general ill-posed as it may not be feasible due to the constraints (2).These challenges are related to the design of reference governors (Garone et al., 2017), a control unit that adjusts an externally provided reference y d before passing it to a lower level controller in order to ensure constraint satisfaction.Chisci and Zappa (2003); Chisci et al. (2005) propose a feasibility recovery mode, which similarly adjusts the reference before passing it to an MPC to ensure feasibility.External loops to adjust the reference in MPC schemes are also used by Mayne and Falugi (2016) and Skibik et al. (2021Skibik et al. ( , 2022)).The following exposition utilizes the framework of artificial references, which includes this reference computation in the MPC formulation.This idea was initially developed by Limón et al. (2008) 6 for linear systems and has led to many further developments in the literature concerning: setpoint tracking for linear systems (Ferramosca et al., 2009(Ferramosca et al., , 2010;;Simon et al., 2014;Zeilinger et al., 2014b;Limón and Alamo, 2015;Berberich et al., 2020b;Aboudonia et al., 2022), extensions to nonlinear systems (Limón et al., 2018;Köhler et al., 2020b;Cotorruelo et al., 2020;Berberich et al., 2022a;Soloperto et al., 2022;Köhler et al., 2022;Cunha and Santos, 2022;Köhler et al., 2023a;Rickenbach et al., 2023;Galuppini et al., 2023) and periodic reference trajectories (Limón et al., 2014(Limón et al., , 2016;;Köhler et al., 2020b;Yang et al., 2021;Köhler et al., 2023b).

Setpoint tracking MPC using artificial references
For a given state x ∈ X and a target value y d ∈ Y, the setpoint tracking MPC formulation is given by min A minimizer is denoted by r ⋆ (x, y d ), u ⋆ (x, y d ) with the value function J ⋆ tr (x, y d ) and the closed-loop system is given by The main difference to a standard MPC formulation is the joint optimization over the artificial reference r = (x r , u r ) ∈ Z r that minimizes the offset cost V o .For a fixed artificial reference r, Problem (14) corresponds to a standard stabilizing MPC scheme as studied in Theorem 1.The external target y d only appears in the cost V o and hence feasibility can be guaranteed independent of y d .We consider ℓ(x, u, r) and terminal ingredients V f , X f designed according to Section 3.2 or Section 3.5.2,which satisfy Assumption 2 for any r ∈ Z r .In addition, these designs locally 7 ensure feasibility of Problem (4) and a quadratic upper bound on the value function J ⋆ N (x), cf.Köhler et al. (2020b, Prop. 4, Lemma 5).Theorem 3. Suppose Problem (14) is feasible for x = x 0 .Then, for any time-varying target sequence y d (t), Problem (14) is feasible for all t ∈ I ≥0 and the constraints (2) are satisfied for the resulting closed-loop system (15).Furthermore, if y d is constant, then lim t→∞ x ⋆ r (x(t), y d ) − x(t) = 0.The proof is analogous to Theorem 1 by using the previously optimal artificial reference r ⋆ as a candidate solution.

Stability analysis
Theorem 3 only ensures convergence to the artificial steady state x ⋆ r .Convergence and stability w.r.t. the target y d require additional conditions.We consider a quadratic offset cost V o (y, y d ) = y − y d 2 S with S ≻ 0, but much of the presented theory generalizes to continuous (sub)differentiable, strictly convex functions (Limón et al., 2018, Asm. 2).The set of feasible output references is given by and a minimizing setpoint is given by y (Limón et al., 2018, Asm. 1-2) (Tr.1)Convexity: The set Y s is convex.
(Tr.2) Uniqueness: There exists a unique Lipschitz continuous function g : Y s → Z r , such that g(y) = (x, u) for any steady state (x, u) ∈ Z r , f (x, u) = x, y = h(x, u).
Assumption 4 and V o (strictly) convex imply that there exist a unique optimal steady state (x ⋆ rd , u ⋆ rd ) = g(y ⋆ rd ) and the following theorem ensures stability of this steady state.
Theorem 4. (Köhler et al., 2020b, Thm. 8) Suppose Problem (14) is feasible for x = x 0 and y d (t) is constant for all t ∈ I ≥0 .Then, the optimal steady state x ⋆ rd is exponentially stable for the resulting closed-loop system (15) with the Lyapunov function J ⋆ tr (x, y d ) − V o (y ⋆ rd (y d ), y d ).The main idea behind this result is that if x is close to the artificial reference x ⋆ r , then the optimizer can move the artificial reference y r towards the minimizer y ⋆ rd (using convexity), thus also decreasing the offset cost V o .Limón et al. (2008Limón et al. ( , 2018) ) showed convergence with a case distinction, however, more recent results directly derive suitable Lyapunov inequalities (Zeilinger et al., 2014b;Limón et al., 2014Limón et al., , 2016;;Köhler et al., 2020b;Soloperto et al., 2022;Cunha and Santos, 2022;Köhler et al., 2023a).The following intermediate lemma provides further intuition for this stability result.
Lemma 1. (Adapted from (Soloperto et al., 2022, Prop. 2)) There exists a constant c > 0, such that for any (x, y d ) with Problem (14 7 For any x − x r ≤ ǫ with some uniform ǫ > 0. A similar bound is derived by Limón et al. (2014, Lemma 2) assuming continuity of minimizers w.r.t.x and results comparable to Lemma 1 are also recently derived by Cunha and Santos (2022, Lemma 4) and Köhler et al. (2023a, Lemma 2).Lemma 1 ensures that the distance between the artificial reference r ⋆ and the optimal setpoint r ⋆ rd = g(y ⋆ rd ) is upper bounded by the distance of the system state x to the artificial reference x ⋆ r , which converges to 0 according to Theorem 3.

Discussion
In the following, we discuss some of the properties of this setpoint tracking MPC formulation and different variations.

General properties
The introduction of the artificial setpoint r in Problem ( 14) does not increase the design complexity and, depending on the prediction horizon N, has a small impact on the overall computational complexity.However, there are a number of practical benefits (cf.Sec.7.3.1).Feasibility is completely independent of the external target y d .Having a unified formulation that jointly solves planning and tracking simplifies the design and the offset cost V o allows for intuitive tuning, e.g., Ferramosca et al. (2009) ensure local optimality using a large (exact) penalty while Berberich et al. (2022a) provide strong robustness properties using a small offset cost.Notably, artificial references can also significantly increase the region of attraction, enabling large setpoint changes with arbitrary small prediction horizons.Artificial setpoints also provide a simple tool to enable coordination in a distributed setting (Köhler et al., 2022;Rickenbach et al., 2023) and can easily accomodate online model updates (Berberich et al., 2022a;Sasfi et al., 2022;Peschke and Görges, 2023).

Convexity
The presented analysis (Thm.4/Lemma 1) uses convexity of the set of feasible steady-state outputs Y s .This condition does not require convexity of the steady-state manifold (x, u), which can be significantly more restrictive for nonlinear systems (cf.Köhler et al. (2020b, Fig. 1)).If Y s is not convex, the closedloop system converges to some steady state, which is in general a local minimum.Soloperto et al. (2023) show that this issue can sometimes be circumvented by simply increasing the offset cost V o and using a large horizon N.More systematically, Cotorruelo et al. (2020) derive a transformation of the output to ensure the convexity condition holds for the special case of nonconvex sets in normal form (e.g., star-shaped).Soloperto et al. (2022, Prop. 4) show that convexity is not needed if the offset cost V o is chosen as the path-distance restricted to the feasible steady-state manifold.The design by Cotorruelo et al. (2020) allows for an efficient implementation, but application for more general environments with multiple obstacles seems challenging.The approach by Soloperto et al. (2022) is applicable to complex non-convex environments.However, the implicit characterization of V o essentially requires the solution to a (simplified) planning problem, thus increasing computational complexity.Further research is required to enhance applicability for cluttered obstacle domains in robotics.

Uniqueness and zone tracking
Condition (Tr.2) in Assumption 4 ensures that the optimal steady state x rd is unique.Limón et al. (2018, Rk. 1) derive a unique Lipschitz continuous function g using a rank condition on the Jacobians of f, h and the implicit function theorem.If this condition does not hold, one can also specify a unique map g in Problem ( 14), cf.(Limón et al., 2018, Equ. (9)).More generally, if stability of a specific point is not required, a more flexible zone tracking MPC can be considered (Ferramosca et al., 2010;Soloperto et al., 2022).In this case, the offset cost V o can, e.g., penalize deviations of the output y from some general desired set Y d ⊆ Y. Correspondingly, one can show that the closed-loop system stabilizes a set of optimal steady states instead of one unique state.Notably, while the analysis, e.g., to show Lemma 1, can become more complex (Soloperto et al., 2022, Prop. 1)), the implementation of such a zone-tracking MPC remains simple.8

Semidefinite input-output cost ℓ
For the intermediate result (Lemma 1) and for the overall stability result (Thm.4), it is crucial that the stage cost ℓ is positive definite w.r.t.x.However, e.g., in case of input-output models, it is quite common to formulate the stage cost ℓ using the output y, which is only positive semidefinite in the state x.Berberich et al. (2020b) show stability of the optimal steady state x ⋆ rd with such stage costs for linear observable systems.Galuppini et al. (2023) provide initial results for more general nonlinear detectable systems.However, in contrast to Berberich et al. (2020b), this implementation explicitly requires the storage function certifying detectability.This is due to a difference in the considered Lyapunov function, with the storage function evaluated at the online optimized steady state x r or the (unknown) optimal steady state x ⋆ rd .Developing an approach that merges the benefits of these two formulations is an open issue.4.2.5.Terminal set X f (r) for setpoint tracking Limón et al. (2008) provides a polytopic terminal set X f (r) using the maximal admissible set of an augmented state for linear systems with polytopic constraints.However, computing this set can result in scalability issues and this approach does not transfer to nonlinear systems.Instead, a common approaches is to scale a parametrized set:  2021) obtain a polynomial expression for α(r) by solving a sum-of-squares (SOS) problem offline.However, computing a parametrized function α(r) offline is conservative.Köhler et al. (2020b) address this issue by including the scaling α ∈ [α min , α 1 ] ⊆ R ≥0 as a decision variable in Problem ( 14).Specifically, the constraint r ∈ Z r in Problem ( 14) is replaced by comparable constraints on (r, α) that corresponds to Condition (T.1).In the special case of k f /V f linear/quadratic (cf.Sec. 3) and Z polytopic, this is a set of linear inequality constraints.However, the approach can also be applied if Z is given by Lipschitz continuous inequality constraints (Köhler, 2021, Sec. 3.2.2).The lower bound α ≥ α min > 0 ensures that Lemma 1 holds with a uniform constant c > 0 and hence the exponential stability result (Thm.4) remains valid (Köhler et al., 2020b, Prop. 12).The increase in computational demand due to the online optimization of the scaling α tends to be negligible, while it can have significant performance benefits (Köhler et al., 2020b, Sec. 4).For linear systems with a polytopic terminal set X f , a comparable online optimization of the scaling α is also suggested by Simon et al. (2014).It is possible to further generalize the terminal set by online optimizing over matrices P ∈ R n×n , K ∈ R m×n characterizing V f , k f (Yang et al., 2021;Aboudonia et al., 2022).However, this introduces LMI constraints in the MPC formulation, which can significantly increase the computational complexity.

Dynamic reference tracking
Section 4.2 ensures recursive feasibility for arbitrary timevarying targets y d (t), but stability/performance guarantees are restricted to (piece-wise) constant targets y d .Intuitively, online optimized artificial setpoints r provide reasonably good performance if the target y d is only "slowly" changing with time.9However, in many applications the optimal operation is intrinsically dynamic (C.1) and trying to steer the system to a steady state r ∈ Z r may not result in satisfactory performance.Instead, similar to Sections 3.3-3.4,dynamic reference trajectories r k ∈ Z r , k ∈ I ≥0 need to be considered.

Periodic reference tracking
The special case of periodic target signals, i.e., y d (t + T ) = y d (t) with some period length T ∈ I ≥1 , has been studied for linear systems by Limón et al. (2014Limón et al. ( , 2016) ) with a minimizer r ⋆ rd .Given the state x ∈ X and target signal y d ∈ Y T , the periodic tracking MPC formulation is given by: min Problem ( 18) jointly optimizes a periodic artificial reference r ∈ Z T r to match the target signal y d ∈ Y T and computes an input sequence u ∈ U N to drive the system state x to this artificial reference r ∈ Z T r .For T = 1, this reduces to the setpoint tracking MPC in Problem ( 14).For a fixed artificial reference r, Problem (18) reduces to the trajectory tracking MPC scheme studied in Theorem 2. Assuming the terminal set X f and the terminal cost V f are properly designed (cf.Sec.3.3, 3.5.2),this scheme inherits the same properties of Theorem 3: (i) Recursive feasibility and constraint satisfaction holds for any e., the system converges to the artificial periodic reference r.Similar to Theorem 4, stability of the optimal periodic reference trajectory can be guaranteed with a suitable uniqueness and convexity condition for r ∈ R T , see Köhler et al. (2020b, Thm. 8) for details.

Discussion
This periodic tracking MPC scheme shares all the theoretical properties of the setpoint tracking MPC (Sec.4.1), enabling a seamless extension to dynamic/periodic problems.However, there are some practical concerns to be considered: (i) Computational complexity is increased, especially if T ≫ N.This issue will be addressed in Section 4.4.(ii) The design of terminal ingredients V f , X f is more challenging.One solution is to combine the parametrized terminal ingredients (Sec.3.4) with an online optimized scaling α > 0 (Sec.4.2.5)(Köhler et al., 2020b, Prop. 12).Other possibilities include simple terminal equality constraint (Köhler et al., 2020b, Prop. 4) or online optimizing terminal ingredients (Yang et al., 2021).(iii) There are numerous control problems which cannot be addressed using periodic references r ∈ Z T r and target signals y d ∈ Y T .General time-varying targets y d remain an open problem.

Partially decoupled tracking and planning
The MPC formulations with artificial references provides a general framework, combining a tracking MPC (Problem (10)) with reference planning (Problem (17)).However, there are different cases in which the reference planning problem becomes computationally expensive: periodic references r ∈ Z T r with T ≫ N (Sec.4.3); non-trivial offset cost V o (Soloperto et al., 2022, Prop. 4), (Rickenbach et al., 2023, Sec. 5); joint optimization of terminal ingredients (Yang et al., 2021;Aboudonia et al., 2022).This increases the computational complexity of the MPC scheme, which can be a bottleneck for practical application.

Decoupled tracking & planning
A natural solution is to split Problem ( 14) in two problems: a tracking MPC and a reference planner that can be solved independently.This corresponds to a classical tracking-planning decomposition as often applied with some tracking controller (not necessarily MPC), see the overview by Schweidel et al. (2022).Corresponding approaches where the tracking controller is given by an MPC scheme (Sec.3.3) are also routinely applied, see the architecture by Mayne et al. (2011) or experimental results by Liniger et al. (2015); Romero et al. (2022).Online changes in the target y d require online re-planning of the reference r.This may yield feasibility issues in the tracking MPC scheme if the reference planning is completely decoupled from the tracking.Köhler et al. (2020b, Sec. 3.4) address this issue by updating the reference every M ∈ I ≥1 steps and imposing additional constraints on the planner to ensure feasibility of the tracker.For simplicity, we consider again the special case of T -periodic reference trajectories (Sec.4.3).At time

Constrained reference planning
The set RT characterizes the set of periodic reference trajectories r ∈ Z T r with scaling variables α ∈ R T ≥0 such that the periodic sequence of terminal sets X f (r k , α k ) satisfies Assumption 3 (Köhler et al., 2020b, Equ. ( 29b)-( 29e)). 10In the absence of constraint (19c), Problem (19) corresponds to a decoupled reference planning.Equation (19c) constrains the planner such that feasibility of the terminal set constraint is preserved, see Figure 1 for an illustration.Condition (19c) is in general not computationally tractable, but simple sufficient conditions can be obtained using the quadratic parametrization of V f from Section 3.4 (Köhler et al., 2020b, Prop. 15).A feasible candidate solution to Problem ( 19) is given by the previous solution shifted by M steps.
This design directly ensures recursive feasibility of the tracking MPC and the planner.However, Condition (19c) may prevent convergence to the optimal periodic trajectory, i.e., in general (19c) might only be feasible if r N = r N+M (t i ).Köhler et al. (2020b, Prop. 16) address this issue by using an exponential decay of the CLF V f with some known factor ρ < 1 to implement Figure 1: Illustration how the constrained reference planning (19) ensures recursive feasibility of the tracking MPC at time t i+1 .Closed-loop state x(t), t ∈ I [t i ,t i+1 ] (blue, solid), predicted state sequence x u of the tracker (blue, dashdotted); artificial reference r at time t i (red, dashed) with terminal set scaling α (red-dotted); artificial reference r at time t i+1 (magenta, dashed) with terminal set scaling α (magenta, dotted).The predicted state sequence x u of the tracker (blue, dash-dotted) satisfies the new terminal set constraint (magenta), since it contains the previous terminal set constraint (red), cf. (19c).
a contractive terminal set.This ensures that Constraint (19c) is inactive for small enough reference updates and hence finitetime convergence of r to the optimal periodic reference can be ensured.

Partial coupling between tracking and planning
Problem ( 19) uses no information of the current state of the tracker, which can result in slow convergence of the reference.Köhler et al. (2020b, Alg. 1) propose a partial coupling, which utilizes the most up to date information of the tracker to relax Condition (19c).Specifically, at time t i we know the predicted terminal cost in the tracking MPC, which satisfies V f (x u ⋆ (N, x(t i )), r N (t i )) ≤ α N (t i ).Thus, we shrink the scaling α N+k (t i ), k ∈ I ≥0 to enforce more restrictive (but feasible) contractive constraints for the tracking MPC.As a result, Constraint (19c) can be relaxed to The reduction in conservatism is due to the difference between the achieved convergence of the closed-loop tracking MPC and the a priori worst-case bound based on the local contraction rate ρ < 1 of the terminal control law k f .Skibik et al. (2022) propose a similar strategy for the special case of linear systems, steady states (T = 1), and synchronous updates (M = 1).

Discussion
Overall, this partially decoupled strategy retains the theoretical properties of the MPC formulations with artificial references: recursive feasibility and convergence to the optimal feasible reference.A standard tracking MPC scheme is implemented and reference updates are computed in parallel on a different time scale (M ≫ 1).Notably, the additional constraint (20) to recompute the reference r(t i+1 ) in the planner (Problem ( 19)) depends on the reference r(t i ) and the state x(t i ), which are available at time t i .11Hence, this more complex planning problem can be solved in the time interval [t i , t i+1 ].This significantly reduces the computational requirements, allowing for fast feedback and complex planning problems in practical application.However, an increase in M also slows down convergence to the optimal periodic reference y ⋆ rd , resulting in a trade-off, see the numerical comparison by Köhler et al. (2020b, Sec. 4).Practical application for non-periodic planning problems requires further research.

Economic MPC
In this section, we extend the problem formulation to economic MPC (Müller and Allgöwer, 2017;Ellis et al., 2017;Faulwasser et al., 2018).In control theory, one of the primary goals is to achieve stability of the closed-loop system and steer the system state/output to some desired setpoint or trajectory.In Sections 2-4, we accomplished such tracking objectives with MPC schemes by using a stage cost ℓ which is positive definite w.r.t. a desired mode of operation (cf.Asm. 1).In economic MPC this condition is removed and instead an economic stage cost ℓ e is used that directly reflects the desired objective (C.3), e.g., minimizing energy consumption in HVAC or maximizing the yield in a chemical plant.Correspondingly, in this section our primary goal is not to ensure some form of stability.Rather, we derive bounds on the closed-loop performance in terms of the economic stage cost ℓ e .We first discuss how the basic stabilizing MPC can be extended to economic MPC (Sec.5.1), including non-stationary operation (C.1) using a periodic reference (Sec.5.2).Then, we derive an economic MPC scheme for online changing operating conditions (C.2) using artificial setpoints (Sec.5.3) and periodic artificial references (Sec.5.4).Finally, we discuss convergence/stability in economic MPC (Sec.5.5) and the design of an economic terminal cost (Sec.5.6).

Performance guarantees in economic MPC
In the following, we extend the standard stabilizing MPC scheme from Section 2 to directly consider an economic stage cost ℓ e .Our only assumption on the stage cost ℓ e (x, u) is continuity, which in combination with Z compact ensures that ℓ e is bounded.We define an optimal steady state (x r , u r ) ∈ Z by le = min We consider an economic MPC scheme which directly minimizes the economic stage cost ℓ e over the prediction horizon: The closed-loop operation is defined by where u ⋆ (x) is a minimizer to Problem ( 22).The conditions on the terminal ingredients are adjusted as follows.
Assumption 5. Conditions (T.1)-(T.2) from Assumption 2 hold.Furthermore, the terminal cost V f,e : X → R is continuous and satisfies This condition can be trivially satisfied using a terminal equality constraint w.r.t. the steady state x r , i.e., V f,e = 0, X f = {x r }, k f = u r , and more general designs are discussed in Section 5.6.
Theorem 5. Suppose Problem (22) is feasible with x = x 0 .Then, Problem ( 22) is feasible for all t ∈ I ≥0 , the constraints (2) are satisfied, and the following performance bound holds for the resulting closed-loop system (23): This result was initially developed by Angeli et al. (2012, Thm. 1) with a terminal equality constraint and then generalized by Amrit et al. (2011, Thm. 18).Theorem 5 ensures that the closed-loop performance with an economic MPC is (on average) no worse than simply steering the system to a steady state, as done in a standard tracking MPC scheme.On a theoretical level, the only difference to Problem (4) is that Assumptions 1-2 are relaxed to Assumption 5. From a conceptual level, the more important difference is that the MPC directly minimizes the cost ℓ e instead of indirectly considering it by steering the system to some optimal setpoint x r .Such a direct minimization of the economic criterion ℓ e can significantly enhance performance, cf. the numerical investigations by Rawlings et al. (2008Rawlings et al. ( , 2012)); Ellis et al. (2017); Köhler et al. (2020c) and the discussion in Section 7.3.2.

Periodic economic MPC
The design in Section 5.1 first determines an optimal steady state x r and then Theorem 5 ensures that the closed-loop performance is no worse than operation at this optimal steady state.In the following, we generalize this to provide performance guarantees w.r.t.optimal periodic operation, similar to the works by Zanon et al. (2017) and Alessandretti et al. (2016).We consider an economic stage cost ℓ e (x, u, t) which is time-varying and T -periodic with some period length T ∈ I ≥1 .We compute an optimal T -periodic reference r(t where R T ⊆ Z T is the set of T -periodic references (cf.Sec.4.3.1).The goal is to design an economic MPC scheme which outperforms operation at this periodic reference r(t).For a given state x and time t, the periodic economic MPC formulation is given by: min with a minimizer u ⋆ (x, t) ∈ U N and X f , V f,e are T -periodic.The closed-loop system is given by Assumption 6.There exists a T -periodic terminal control law k f : X × I ≥0 → U such that for all t ∈ I ≥0 and all x ∈ X f (t) Conditions (T.1)-(T.2) from Assumption 3 hold, V f,e is continuous, and Problem (27) is feasible for all t ∈ I ≥0 , the constraints (2) are satisfied, and the following performance bound holds for the resulting closed-loop system (28): Theorem 6 ensures that closed-loop performance with the economic MPC is (on average) no worse than using a trajectory tracking MPC scheme (Sec.3.3) to drive the system to the optimal periodic reference r.The proof of this result is analogous to Theorem 6 (Zanon et al., 2017, Rk. 5.8).Assumption 6 can be satisfied with a simple terminal equality constraint.

Shifted terminal cost
The terminal cost decrease (29) is w.r.t. the cost ℓ e at a specific reference point x r (t), while the performance guarantee (30) is w.r.t.le , i.e., the average performance over one period length.The following proposition shows how we can modify the characterization of the terminal cost V f,e to directly consider le .
Proposition 2. (Köhler et al., 2020c, Lemma 5) For any t ∈ I ≥0 and any x ∈ X f (t), the shifted terminal cost Ṽf,e (x, t) Inequality (31) direct relates the decrease in the terminal cost with le , the desired average cost of the periodic reference r, which becomes crucial for the design later (Sec.5.4).The same design condition is required to show that closedloop performance is better than performance at some general set (Dong and Angeli, 2018).A simple design for the terminal cost V f,e for such more general problems remains an open issue, compare, e.g., Martin et al. (2019) for a specific (non-standard) economic MPC formulation to deal with this issue.

Online changing costs using artificial setpoints
In the following, we consider an economic stage cost ℓ e (x, u, y e ) that depends on some time-varying external parameters y e (t) ∈ Y ⊆ R p , e.g., reflecting changes in online prices or user preference.Online variations in y e (t) result in an online change of the optimal setpoint x r .Ideally, the economic MPC should provide a performance that is is no worse than this new optimal setpoint.Müller et al. (2013Müller et al. ( , 2014a)); Fagiano and Teel (2013) and Ferramosca et al. (2014) propose economic MPC formulations using artificial setpoints (cf.Sec.4.1) to address this issue.We first present the approach by Müller et al. (2013Müller et al. ( , 2014a) ) and then explain the relation and difference to the approaches by Fagiano and Teel (2013) and Ferramosca et al. (2014).For a given state x, parameters y e ∈ Y ⊆ R p , a weight β ≥ 0, and a later specified variable κ ∈ R, the economic MPC formulation is given by + βℓ e (x r , u r , y e ) 32) A minimizer is denoted by r ⋆ (x, y e , κ, β), u ⋆ (x, y e , κ, β) and the closed-loop system is given by κ(t + 1) =ℓ e (r ⋆ (x(t), y e (t), κ(t), β(t)), y e (t + 1)), with some later specified weight β(t), and initialization β(0) = β 0 ≥ 0 and κ(0) = κ 0 = ∞.Compared to the setpoint tracking MPC (Problem ( 14)), Problem (32) directly minimizes the economic cost ℓ e instead of utilizing a positive definite tracking stage cost ℓ.The added constraint involving κ ensures that the cost at the artificial reference r does not deteriorate compared to the previous solution.The weighting β is used to ensure convergence of the artificial reference to a (local) minimum (Sec.5.3.2/5.3.3).

Relative performance guarantees
In the following, we assume that V f,e satisfies the conditions in Assumption 5 for any (constant) feasible setpoint r ∈ Z r ⊆ Z and any (constant) y e ∈ Y. Furthermore, we require the following technical conditions: β(t) is non-negative, asymptotically constant, and β(t + 1) − β(t) admits a uniform upper bound (Müller et al., 2013, Asm. 1), which are trivially satisfied with a constant weight β ≥ 0.
Theorem 7. (Müller et al., 2013, Thm. 1) Suppose Problem (32) is feasible for x = x 0 .Then, for any time-varying parameters y e (t), Problem (32) is feasible for all t ∈ I ≥0 and the constraints (2) are satisfied for the resulting closed-loop system (33).Furthermore, if y e is constant, then the following performance bound holds Theorem 7 ensures that the closed-loop performance is no worse than the performance at the artificial reference, which is given by κ.This result does not imply that the artificial reference r converges to some (local) optimum.In Theorem 4 we address a similar issue for a setpoint tracking MPC scheme, where we use the fact that the system converges close to the artificial reference and hence the artificial reference can be incrementally moved w.r.t. the candidate solution (see also Lemma 1).However, the closed-loop system with the economic MPC will in general not converge close to the artificial setpoint r.Müller et al. (2014a) propose the following solution to this problem: The terminal set constraint X f should have a nonempty interior and be exponentially contractive, e.g., using a terminal set X f designed according to Section 3.2.This ensures that at any point in time, the artificial reference r can be incrementally moved (Müller et al., 2014a, Lemma 1).Then, by suitably increasing the weighting β(t) online if needed, Müller et al. (2014a, Thm. 3) ensure that the closed-loop performance is no worse than operating at a locally optimal steady state.12

Constant weight β and suboptimality bound
The above discussed work by Müller et al. (2013Müller et al. ( , 2014a) is inspired by Fagiano and Teel (2013).Instead of ensuring convergence of the artificial reference r by increasing the weight β, Fagiano and Teel (2013, Prop.2) use a large constant weight β and derive a suboptimality estimate ǫ(β) w.r.t. a local minimizer.Furthermore, Fagiano and Teel (2013, Alg. 3) use simple terminal equality constraint in combination with a multi-step implementation.135.3.4.Linear systems and strong duality Ferramosca et al. (2014) address the problem in the special case of linear dynamics using a strong duality assumption.In particular, a shifted economic stage cost is minimized and a sufficiently large positive definite offset cost V o (Sec.4.1) is utilized to ensure asymptotic stability of the optimal steady state.However, these techniques cannot be transferred to the nonlinear setting and the minimized cost does in general not have an intuitive "economic" interpretation.

Periodic optimal operation using artificial references
In the following, we combine the designs and problem setups from Sections 5.2 and 5.3, following the work by Köhler et al. (2020c).Specifically, we consider a stage cost ℓ e (x, u, y e ), which depends on some parameters y e (t) ∈ Y.If y e is Tperiodic, then we wish to provide performance guarantees w.r.t. the optimal T -periodic operation.However, the parameters y e may also be subject to additional unpredictable fluctuations and we need to guarantee reliable operation during such transient phases.Such problems naturally arise, e.g., in water distribution networks, electrical networks or HVAC systems, where the optimal mode of operation depends largely on external variables (e.g., the weather).These variables have a strong periodic component (e.g., due to the day-night cycle) but are also subject to additional variations that are difficult to predict.

Economic MPC using periodic artificial references
A natural solution to extend the economic MPC formulation in Section 5.3 to periodic problems is to replace the artificial setpoint r ∈ Z r by a periodic reference trajectory r ∈ R T ⊆ Z T r and consider a periodic prediction y e ∈ Y T for the parameters with y e,0 (t) = y e (t).For a given state x, time t, parameters y e ∈ Y T , weight β ≥ 0, and variable κ ∈ R, an economic MPC formulation with periodic artificial references is given by: min where we define r k := r mod(k,T ) , y e,k := y e,mod(k,T ) for k ∈ I ≥T .
A minimizer is denoted by r ⋆ (x, y e , κ, β), u ⋆ (x, y e , κ, β) and the closed-loop system is given by , y e (t), κ(t), β(t)), y e,k (t + 1)), with initialization κ(0) = κ 0 = ∞ and a self-tuning weight β(t) ≥ 0. This is a direct extension of the approach in Section 5.3, replacing the artificial setpoint r ∈ Z r by an artificial periodic reference r ∈ Z T r .

Pitfalls -periodic artificial references
Next, we provide a negative result: A direct/naïve extension of the design in Section 5.3 to the periodic setting does not provide the desired performance guarantees.We consider the following standard conditions for the terminal ingredients.
Assumption 7.There exists a control law k f : X × R T , such that for any periodic reference r ∈ R T and any state x ∈ X f (r) and any y ∈ Y T : For a given periodic trajectory r and parameters y e , Assumption 7 corresponds to Assumption 6 and is trivially satisfied by a terminal equality constraint X f (r) = {x r,N }, V f,e = 0. Theorem 8. (Köhler et al., 2020c) Suppose Problem (35) is feasible for x = x 0 .Then, for any parameters y e (t), Problem (35) is feasible for all t ∈ I ≥0 and the constraints (2) are satisfied for the resulting closed-loop system (36).Furthermore, if y e is consistently periodic, i.e., y e (t + k) = y e,k (t), t, k ∈ I ≥0 , then the following performance bound holds , y e,0 (k)).
Theorem 8 ensures recursive feasibility and constraint satisfaction independent of the parameters y e .The performance bound (37) follows from Condition (E.3) of the terminal cost V f,e .However, this bound is not w.r.t. the average cost of the periodic reference r ⋆ , but only w.r.t.r ⋆ 0 , the first point of the periodic reference.Köhler et al. (2020c, Sec.II.C) provide a simple example (inspired by the counter example by Müller and Grüne (2016)) that demonstrates that the closed-loop performance can be arbitrarily bad compared to the periodic reference r ⋆ .The problem is that the sequence of optimized trajectories r ⋆ 0 (t), t ∈ I ≥0 can differ completely from the periodic artificial reference r ⋆ ∈ Z T r , even if the average cost of the periodic trajectories coincide.This shifted terminal cost directly provides a decrease condition w.r.t. the average performance at the periodic reference.

Performance guarantees
The following theorem demonstrates that this yields the desired performance guarantees.
Theorem 9. (Köhler et al., 2020c, Prop. 3, Prop. 6, Cor. 1) Suppose the conditions in Theorem 8 hold and we replace the terminal cost V f,e in Problem (35) by Ṽf,e .Then, the closed-loop system (36) satisfies the following performance bound Furthermore, if β(t) is updated using (Müller et al., 2013, Update scheme 2 or 6) and the terminal set X f is contractive (Köhler et al., 2020c, Asm. 3), then the closed-loop performance is on average no worse than the performance at a locally optimal periodic orbit.
Theorem 9 shows that we obtain the desired performance guarantees, outperforming standard stabilization of some optimal periodic trajectory.The design of this shifted terminal cost Ṽf,e is motivated by the characterization of periodic optimality by Köhler et al. (2018a, Prop. 1), which is inspired by non-monotonic Lyapunov functions (Ahmadi and Parrilo, 2008).Unfortunately, the shifted terminal cost Ṽf,e does not admit an intuitive interpretation.Köhler et al. (2020c, Prop.2) also report an alternative design ensuring the performance bound (38) by constraining ℓ e (r k , y e,k ), k ∈ I [0,T −1] .However, this requires an additional continuity condition to ensure local optimality (Köhler et al., 2020c, Asm. 5).

Periodicity constrained economic MPC
Houska and Müller (2017); Wang et al. ( 2018) develop a periodicity constrained economic MPC scheme that intrinsically provides closed-loop performance bounds w.r.t. the optimized periodic reference.Instead of separately optimizing a predicted input sequence u ∈ U N and a periodic trajectory r ∈ R T , this approach only optimizes over periodic trajectories that starts at the current state x: Problem (39) can be recovered as a special case from Problem (35) by setting the prediction horizon N = 0 and using a simple terminal equality constraint X f = {x r,0 }, V f,e = 0.This special case does not satisfy the technical conditions we posed on the terminal set X f (Köhler et al., 2020c, Asm. 3).Hence, Theorem 9, which shows convergence to a locally optimal reference, is not applicable.In fact, convergence to a locally optimal periodic orbit can only be established under rather restrictive conditions, cf.Wang et al. (2018, Example 6, Thm. 1) and Houska and Müller (2017, Lemma 3, Thm. 4).Köhler et al. (2020c, Sec.V.B, App.A) also provide a numerical comparison, demonstrating that the lack of a prediction horizon N limits the performance of this approach.

Stability and convergence in economic MPC
While Sections 5.1-5.4focus on performance guarantees, the question of stability and convergence can also plays an important role when studying economic MPC schemes.

Dissipativity, optimality, stability
Dissipativity plays a crucial role to study stability in economic MPC (Faulwasser et al., 2018) and optimal control (Grüne, 2022).Neglecting some technical conditions, one can show equivalence between the following properties: • the system is optimally operated at the steady state x r (in a strict sense); • (strict) dissipativity w.r.t. the supply rate s(x, u) = ℓ(x, u)− ℓ(x r , u r ), i.e., ∃λ : • the closed-loop system (23) (asymptotically) stabilizes x r ; • the turnpike property holds, i.e., optimal solutions stay close to x r most of the time; see Angeli et al. (2012); Grüne and Müller (2016); Müller et al. (2015); Faulwasser et al. (2017Faulwasser et al. ( , 2018)); Müller (2021) for details.Loosely speaking, these results imply that the economic MPC scheme "does the right thing": If operation at the steady state is optimal, then the closed-loop will converge to this steady state.If the closed-loop system does not stabilize the steady state, then superior performance is achieved using some dynamic operation.Generalization of these concepts are available for periodic orbits (Zanon et al., 2017;Köhler et al., 2018a), general sets (Dong and Angeli, 2018), and the constructive verification of dissipativity is studied (Pirkelmann et al., 2019;Berberich et al., 2020a), cf.Müller (2021) and Grüne (2022) for recent overviews.While dissipativity is not used for the designs in this section, the analysis in Section 6 needs this property.

Enforcing convergence/stability in economic MPC
In various applications, we might wish to constructively enforce convergence/stability to some mode of operation, irrespective of its optimality in terms of the stage cost ℓ e .One can simply use a tracking MPC formulation (Sec.4) to stabilize the optimal steady state or periodic orbit (Limón et al., 2014).Furthermore, it is possible to choose a tracking stage cost ℓ, which locally approximates an economic MPC scheme (De Schutter et al., 2020).In the special case of linear systems and (strictly) convex cost ℓ e , an economic MPC behaves like a stabilizing MPC due to duality/dissipativity, and thus stability can be naturally established (Ferramosca et al., 2014;Broomhead et al., 2015).Alamir and Pannochia (2021) ensure asymptotic stability of the optimal steady state by adding a large enough penalty on non-stationary operation.Gutekunst et al. (2020) achieve convergence to the optimal periodic trajectory by penalizing the non-periodicity in the economic stage cost ℓ e .Convergence/stability can also be imposed with Lyapunov-based constraints (Heidarinejad et al., 2012), (Ellis et al., 2017, Sec. 4), average constraints (Müller et al., 2014b,c;Rosenfelder et al., 2020), or by considering a multiobjective formulation (He et al., 2015), (Faulwasser et al., 2018, Sec. 9), (Soloperto et al., 2020;Eichfelder et al., 2022).

Design of terminal ingredients in economic MPC
In the following, we discuss how to modify the design of the terminal ingredients from Section 3 to satisfy the conditions in economic MPC (Asm.5, 6, 7).A recent overview for these design methods can be found in Köhler et al. (2020c, Sec.IV.A).
The conditions on the terminal set X f , i.e., positive invariance and constraint satisfaction, are equivalent to the conditions needed in the tracking MPC schemes (Sec.3-4) and hence the terminal set can be constructed using the design procedures in Section 3. Recall that the economic MPC schemes with artificial references (Sec.5.3-5.4)require a contractive terminal set, while a simple terminal equality constraint requires modifications to ensure convergence to a local minimum.

Terminal cost -standard design
The conditions on the terminal cost V f,e involve the economic stage cost ℓ e , which is in general neither quadratic nor positive definite.This necessitates modifications to the design procedures in Section 3. Let us first consider a fixed optimal steady state x r (Sec.5.1).Amrit et al. ( 2011) propose a simple design to compute an economic terminal cost of the form V f,e (x) = x 2 P + p ⊤ x that satisfies (24).Specifically, one can derive a local linear-quadratic upper bound on the stage cost ℓ e using the Jacobian and Hessian of ℓ e (Amrit et al., 2011, Lemma 23).Then, the matrix P is designed using a Lyapunov equation comparable to LQR 0 (8), where the role of Q, R is replaced by a positive semidefinite matrix that upper bounds the Hessian of ℓ e .The vector p ∈ R n is chosen such that it exactly compensates the Jacobian of the stage cost ℓ e .Then, analogous to Proposition 1, Inequality (24) holds in a sufficiently small terminal set (Amrit et al., 2011, Lemma 24).Notably, this terminal cost V f,e is not positive definite w.r.t.x r .This captures the fact that, for some initial conditions in the terminal set, the transient cost is strictly smaller than the cost at the optimal steady state.Due to this, the terminal set X f cannot be chosen as a sublevel set of the economic terminal cost V f,e (as done in tracking MPC, Sec.4.2.5).Note that there is no degree of freedom in the choice of p as it needs to exactly cancel the Jacobian of ℓ e .
For periodic reference trajectories (Sec.5.2), we consider V f,e (x, t) = x − x r (t) 2 P(t) + (x − x r (t)) ⊤ p(t).Time-varying matrices P(t) can be computed using a time-varying (periodic) LQR, similar to (Aydiner et al., 2016).Köhler et al. (2020c, Prop. 4) provide a formula to compute p(t) by solving a linear system of equations that includes the Jacobian of the dynamics f and the stage cost ℓ e along the periodic reference r(t).
To account for artificial setpoints (Sec.5.3), the main challenge is related to the linear correction factor p. Specifically, while the approach from Section 3.2 can be used to compute a parametrized matrix P(r), this is in general not possible for p(r).Instead, we include the vector p ∈ X as a decision variable in the MPC that needs to satisfy an equality constraint involving the Jacobian of the dynamics f and the stage cost ℓ e , cf.Müller et al. (2014a, Rk. 7).
For periodic artificial reference trajectories (Sec.5.4) we simply combine the previous two approaches.A parametrized matrix P(r) is computed offline following the design in Section 3.4.A periodic correction term p ∈ X T is included in the online optimization with suitable constraints involving the Jacobian of f and ℓ e , cf.Köhler et al. (2020c, Sec.IV.A).

Positive definite terminal cost
The computation of the correction factor p can become complex, e.g., when considering artificial references.Alessandretti et al. (2016) propose a positive definite terminal cost V f,e by suitably scaling a tracking terminal cost V f , which does not require this correction term p.The same approach can be applied to general artificial references using the parametrized terminal cost (Sec.3) to provide a terminal cost for artificial references without the need to online compute the correction term p (Köhler et al., 2020c, Rk. 5).However, this neglects the gradient information captured by p and is hence more conservative, see also the comparison by Köhler et al. (2020c, App. A).

MPC without terminal constraints/cost
In this section, we analyse simpler MPC formulations, which do not rely on a terminal cost V f or a terminal set X f satisfying Assumption 2. First, we provide some motivation and historical context regarding the analysis of such MPC schemes (Sec.6.1).Then, we derive qualitative conditions in terms of the stage cost ℓ and intrinsic system properties, such that a sufficiently long prediction horizon N ensures desired closed-loop properties (Sec.6.2).Finally, we provide quantitative bounds for a sufficiently long horizon N and discuss how to use these analysis results constructively in the design (Sec.6.3).

Motivation and historical context
In the following, we study an MPC formulation given by with a minimizer u ⋆ (x) and the closed-loop system Compared to the standard formulation in Problem (4), we simply remove the terminal cost V f and the terminal set constraint X f in Problem (40).
The design and analysis of the MPC formulations in Sections 2-5 revolve around the terminal ingredients.However, the design of the terminal ingredients (Sec.3) can be complicated and application to large scale systems brings additional challenges (cf.Sec.7.2.2).This design also requires stabilizability of the linearization, which is not given for non-holonomic systems like a car (Worthmann et al., 2015).Furthermore, the terminal set constraint X f can lead to feasibility issues, e.g., if the design is done incorrectly or the model mismatch is neglected.In contrast, Problem (40) is strikingly simple from an application perspective.Due to these reasons, many MPC implementations do not utilize properly designed terminal ingredients, cf.Section 7.3.3for a balanced discussion regarding drawbacks and merits of terminal ingredients in MPC.

What can go wrong?
For N → ∞, Problem (40) results in the infinite horizon optimal controller, which is known to be stabilizing under standard conditions.Hence, one may expect that the stability properties of the closed-loop system (41) improve if we increase the prediction horizon N.However, stability is in general not monotone w.r.t. the horizon N, i.e., the closed-loop system (41) may become unstable if the prediction horizon N increases, see, e.g., the numerical examples by Primbs et al. (1999, Sec. 4.2) and Köhler et al. (2023, Sec. VII.A).Instability due to a finite horizon N can even be observed for simple stable linear systems, e.g., water tanks (Raff et al., 2006) or mass-spring-damper systems (Köhler et al., 2023).Hence, a theoretical analysis is required to a priori judge the stability properties of such MPC formulations.

Stability conditions -historical developments
Historically, MPC emerged from the process control industry without any system theoretic guarantees and terminal ingredients are (mostly) not used, cf.Qin and Badgwell (2003, Sec. V5), Mayne (2013, Sec. 3 (i)).Such MPC formulations are also called unconstrained MPC (Grüne, 2009;Grüne et al., 2010;Reble and Allgöwer, 2012) due to the absence of a stabilizing terminal constraint (despite the presence of input or state constraints).Given that such MPC formulations are routinely applied, there is a significant interest in understanding the theoretical properties.Early works use regularity properties to ensure stability under some sufficiently long horizon N, cf., e.g., the works by Alamir and Bornard (1995) and Jadbabaie and Hauser (2005).However, it is difficult to obtain quantitative bounds regarding the horizon N using this analysis.
In the following, we summarize the main results in these directions.

Qualitative conditions
In the following, we discuss qualitative conditions on the system and the stage cost ℓ, which ensure desired closed-loop properties with a sufficiently long prediction horizon N. We first focus on the standard problem of driving the system to the origin using a positive definite stage cost ℓ (Sec.6.2.1).Then, we extend the problem to a positive semidefinite input-output stage cost ℓ (Sec.6.2.2) and a singular output cost ℓ (Sec.6.2.3).The generalizations to an economic stage cost ℓ e and dynamic problems are studied in Sections 6.2.4 and 6.2.5, respectively.This exposition considers global stability with pure input constraints, i.e., Z = X × U. Section 6.2.6 shows how these arguments naturally extend to state constraints with a suitable region of attraction.

Cost controllability
We first focus on the prototypical problem of stabilizing the origin with 0 = f (0, 0), 0 ∈ U, and a positive definite stage cost ℓ (Asm.1).
Assumption 8. (Grüne, 2012, Asm. 3.5) There exists a constant γ ≥ 1, such that for any x ∈ X and any horizon N ∈ I ≥1 , the value function satisfies A sufficient condition for ( 42) is exponential cost controllability, which is characterized as follows: For any x ∈ X, there exists an input sequence u In general, Assumption 8 does not hold and there may not exist any finite horizon N ensuring asymptotic stability when using a quadratic stage cost ℓ (Müller and Worthmann, 2017).Systematic choices of ℓ can be derived using homogeneity (Coron et al., 2020) and non-holonomic vehicles are explored by Worthmann et al. (2015); Rosenfelder et al. (2023).
Theorem 10. (Grüne and Rantzer, 2008, Thm. 4.5) There exists a constant N ≥ 0, such that for all N > N and all x 0 ∈ X, the closed-loop system (41) satisfies 43) with α N ∈ (0, 1], the origin is asymptotically stable with the Lyapunov function J ⋆ N , and the following performance bound holds Theorem 10 ensures stability and ( 44) provides a relative suboptimality/regret bound 1/α N ∈ [0, ∞) w.r.t. the infinitehorizon optimal solution J ⋆ ∞ .Inequality ( 43) is also called a relaxed dynamic programming inequality.It holds that lim N→∞ α N = 1 and hence optimality is recovered in the limit.
To simplify the exposition, in the following we assume that the nonlinear system is exponentially stabilizable and hence a simple quadratic cost ℓ can be used.

Cost detectability
In many applications, we encounter a cost of the form , where y = h(x) is some lower dimensional output of the system (Sec.4.1).This is particularly relevant if we identified an input-output model, e.g., an impulse response model, from data.Condition ( 42) is not applicable in this case and we require a detectability condition.Assumption 9. (Grimm et al., 2005, SA 3/4) There exist constants γ ≥ 1, γ o ≥ 0, ǫ o > 0 and a storage function W : X → R ≥0 , such that for any horizon N ∈ I ≥1 and for any x ∈ X, u ∈ U: Condition 1) holds if the system is exponentially stabilizable and h is Lipschitz continuous (Köhler et al., 2022, Prop. 2).Condition 2) ensures that lim t→∞ ℓ(x(t), u(t)) = 0 implies lim t→∞ x(t) = 0, i.e., the system is zero-state detectable with the output y.This property is also called input-output-to-state stability (IOSS), W(x) is an IOSS Lyapunov function, and for linear systems this reduces to the standard detectability condition (Cai and Teel, 2008).In case of nonlinear autoregressive models (NARX), a simple quadratic IOSS Lyapunov function W(x) can be constructed (Köhler et al., 2023, Rk. 3).Assumption 8 is a special case of Assumption 9 considering W(x) = 0.

Singular output cost
Next, we consider a pure output stage cost ℓ(x, u) = h(x) 2 Q without any input regularization.Such a cost appears naturally in case no input reference is available, e.g., in the classical output regulation setting (Köhler et al., 2022).Furthermore, studying this singular cost provides insight for the implications of decreasing the input weighting R (Westenbroek et al., 2022).Due to the absence of an input regularization R ≻ 0, the cost detectability condition 2) is more restrictive.Köhler et al. (2022, Prop. 4) show that this cost ℓ satisfies a comparable detectability condition if the system is minimum-phase/outputinput stable (Liberzon et al., 2002), i.e., the zero dynamics (h(x) ≡ 0) are asymptotically stable.Westenbroek et al. (2022, Lemma 3,Thm.3) derive a converses result: If the system is non-minimum-phase, then for any (finite) prediction horizon N, the closed-loop system is unstable if the input regularization R is chosen small enough.Some intuition for these results can be found by relating the MPC with this singular cost ℓ to a highgain controller, which is known to destabilize non-minimumphase systems (Davison and Wang, 1974, Thm. 8).

Economic cost and dissipativity
In the following, we discuss results for an economic stage cost ℓ e (x, u) (Sec.5), which is indefinite (Asm. 1 does not hold).Grüne (2013); Grüne and Stieler (2014) show that a sufficiently long prediction horizon N ensures (practical) stability of the optimal steady state x r , given strict dissipativity (cf.Sec.5.5.1) and a local continuity/controllability condition.The main difference between this strict dissipativity condition and the cost detectability condition 2) is the fact that W(x) in Assumption 9 needs to be positive semidefinite (Höger and Grüne, 2019, Thm. 2).The implementation of this economic MPC scheme (Problem (40)) only requires the specification of the stage cost ℓ e , but no knowledge of the optimal steady state x r , i.e., the closed-loop system "finds" the optimal steady state.

Dynamic operation
In the following, we explain how the results in Sections 6.2.1-6.2.4 can be naturally extended to address the challenges related to dynamic operation (C.1)-(C.3).Most difficulties listed in Section 2.2 are not present due to the simple design in Problem (40).In particular, as no specific design is required for the implementation, this MPC formulation can directly cope with online changes in the mode of operation (C.2).Furthermore, Sections 6.2.2-6.2.4 study output references or economic costs (C.3).Notably, the results for economic stage costs ℓ e also apply to unreachable references (Rawlings et al., 2008).Specifically, suppose the stage cost is positive definite w.r.t.some state x r , but this setpoint is not feasible (due to dynamics or constraints).Then, the closed-loop system (practically) converges to the steady state with the smallest distance to the unfeasible target x r (Köhler et al., 2019;Long and Xie, 2021).Next, we discuss results for non-stationary operation (C.1).Generally speaking, the previously presented results extend to the time-varying case, assuming the corresponding conditions related to cost controllability, cost detectability or dissipativity are suitably adjusted.
Considering the singular output stage cost (Sec.6.2.3), Köhler et al. (2022) apply this analysis to solve the output regulation problem (Isidori and Byrnes, 1990), i.e., an exogenous system generates a time-varying output reference y r (t) that should be tracked.The classical solution to this problem requires the solution to a partial differential equation to determine the regulator manifold, where this tracking objective is achieved.Köhler et al. (2022, Thm. 2) show that a simple MPC formulation (Problem ( 40)) with an output tracking cost ℓ = h(x) − y r 2 Q implicitly steers the system to this regulator manifold, given a minimum-phase condition.Since this manifold is unknown, regularization w.r.t.some reference input u r cannot be directly applied.However, in case the exogenous signal is periodic, it is possible to penalize the non-periodicity in the input.In this case, the minimum-phase condition can be relaxed to a standard detectability/IOSS condition in combination with a technical non-resonance condition (Köhler et al., 2022, Sec. IV).
For the case of time-varying economic stage costs ℓ(x, u, t), Köhler et al. (2019, Thm. 3, Lemma 4) investigate periodic problems, show strict dissipativity, and prove that the closedloop system implicitly tracks the optimal periodic trajectory, see also Long and Xie (2021) for the continuous-time extension.Grüne and Pirkelmann (2020) extend this result to general time-varying problems.These results assume that there exists a unique time-varying optimal trajectory x ⋆ rd (t).However, in a time-invariant setup, the optimal mode of operation may also be non-stationary, cf., e.g., maximizing the yield of a continuous stirred-tank reactor (Bailey et al., 1971).In case the system is optimally operated at a (time-invariant) periodic orbit, Müller and Grüne (2016, Sec. 3) show that the closed-loop performance can be arbitrarily suboptimal, even with an arbitrarily large horizon N. Suitable stability and performance guarantees can be obtained using: a multi-step implementation (Müller and Grüne, 2016); additional symmetry conditions (Köhler et al., 2018a); or discounting in the cost (Schwenkel et al., 2022).Results for more general optimal operation in some (non-periodic) set are not yet available.

State constraints and region of attraction
The previous analysis uses a global stabilizability condition (Asm.8/9).However, this precludes hard state constraints and most unstable systems.Due to its global nature, it is in general also challenging to verify numerically.We address this issue by considering a relaxed local condition and suitably characterize the region of attraction.In the following, we again consider general state and input constraints (2).
Compared to Assumption 8, this relaxes the requirement to hold only locally around the origin.Similar to Proposition 1, if the nonlinear dynamics are twice-continuously differentiable, the linearization at the origin is stabilizable, and 0 ∈ int(Z), then Assumption 10 holds with some c > 0, γ ≥ 1 (Schulze Darup and Cannon, 2015).Hence, Assumption 10 allows for hard state constraints and unstable systems.Due to the presence of hard state constraints, recursive feasibility of Problem ( 40) is not immediately obvious.
Theorem 12. (Boccia et al., 2014, Thm. 4) For any constant V > 0, there exists a horizon N V ≥ 0, such that for any horizon N > N V and any initial condition x 0 ∈ X is feasible for all t ∈ I ≥0 , the constraints (2) are satisfied, the set X V is positively invariant, and the origin is exponentially stable with the Lyapunov function J ⋆ N for the resulting closed-loop system (41).
Theorem 12 reveals a connection between the region of attraction X V and the horizon bound N > N V .Compared to Theorem 1, recursive feasibility is not ensured for any feasible initial condition, but only a subset X V .The proof first establishes a decrease condition on the Lyapunov function J ⋆ N , which ensures positive invariance of the sublevel set X V and thus recursive feasibility of Problem (40), i.e., "stability implies feasibility".This is in contrast to the standard paradigm for MPC with terminal ingredients, i.e., "feasibility implies stability" (Scokaert et al., 1999), which first establishes recursive feasibility independent of any stability properties.Since the stability properties depend on a sufficiently long horizon N, also recursive feasibility can be lost here if the horizon is not sufficiently large.Esterhuizen et al. (2020, Thm. 1) extend Theorem 12 to continuous-time problems.For detectable and singular costs ℓ, Köhler et al. (2022, Thm. 1,Thm.2) analyse the region of attraction as a sublevel set of the Lyapunov function.For economic stage costs, Faulwasser and Bonvin (2015, Lemma 3) show that the turnpike property (Sec.5.5.1)ensures recursive feasibility, cf. also Faulwasser et al. (2018, Sec.IV.2).Köhler et al. (2019, Thm. 3) analyse the region of attraction of an economic MPC scheme using a sublevel set of the (practical) Lyapunov function, compare also the continuous-time extension by Long and Xie (2021, Thm. 3.3).

Quantitative bounds
In the following, we discuss quantitative bounds on the sufficiently long horizon N ensuring desired closed-loop properties.First, we present quantitative, partially tight, bounds on the prediction horizon for the different problem setups (Sec.6.3.1).Then, we outline how the theoretical results can be constructively used in the design (Sec.6.3.2).

Tight horizon bounds
We first consider the regulation problem (Sec.6.2.1) using Assumption 8 and discuss different techniques to obtain N for this problem, see also (Grüne, 2012).
Observe that there always exists a Hence, a simple candidate solution is obtained by appending the finite-horizon optimal solution at this point k, which ensures (43) with α N > 0 for N > N ≈ γ 2 (Grimm et al., 2005).By using the principle of optimality and recursively applying (42), Tuna et al. (2006) and Grüne and Rantzer (2008) show an exponential decay which ensures stability for N > N ≈ 2γ log(γ).Grüne (2009) determine a tight constant α N satisfying (43) using a linear program (LP) and Grüne et al. (2010) derive the analytical solution N ≈ γ log(γ), see also Reble and Allgöwer (2012) for the continuous-time case.Given Assumption 8, this is the least conservative 15 bound s.t.J ⋆ N is a Lyapunov function (cf.( 43)).For detectable stage costs, Grimm et al. (2005, Thm.1-2) derive a simple but conservative bound for N. Köhler et al. (2022, App. A) reduce conservatism by showing an exponential bound similar to (46) with a stronger cost observability condition.Köhler et al. (2023, Thm. 3) derive a tight estimate N using an LP analysis.
For economic stage costs, the results by Grüne (2013); Grüne and Stieler (2014) are largely qualitative and convergence is only ensured up to a remainder term that vanishes as N → ∞.
Considering the region of attraction X V based on local condtions (Thm.12): Boccia et al. (2014) (Köhler et al., 2019, Rk. 5)).Köhler et al. (2019, Thm. 2) improve this bound by dividing the horizon in two segments: After the first N 0 ≈ V/c time steps, Inequality (42) can be invoked for the tail of the open-loop predictions, i.e., k ∈ I [N 0 ,N−1] .As a result, the sufficient horizon can be decomposed as N V = N 0 + N, where N 0 depends linearly on V and N only depends on the local stabilizability (γ).This decomposition is tight in the sense that no theoretical properties can be established for N < V/c (Köhler et al., 2019, Rk. 6).Limón et al. (2006) use a similar technique to establish that optimal solutions implicitly satisfy a terminal set constraint x u (k, x) ∈ X f for all k ∈ I [N 0 ,N] . 15In practice we may still be able to achieve stability with a shorter horizon N ≪ N.This has two reasons: a) This derivation uses J ⋆ N as a fixed Lyapunov function.b) The horizon bound N is valid for all systems that satisfy condition (42), not only the specific considered dynamics.Köhler (2021, Rk. 4.32) merge this technique with the LP analysis by Grüne et al. (2010) to derive an LP analysis characterizing the region of attraction.Köhler et al. (2022, Thm. 1, Thm. 2) show how similar techniques can be used to characterize the region of attraction for detectable and singular stage costs, cf.Köhler (2021, Thm. 4.12, Thm. 4.50) for improved bounds.Köhler et al. (2019, Thm. 3) and Long and Xie (2021, Thm. 3.3) derive a region of attraction for economic stage costs, however, the bound can be very conservative.
In the special case of open-loop stable systems, Köhler et al. (2023, Rk. 4) show that for any horizon N ∈ I ≥1 , asymptotic stability can be ensured by choosing a sufficiently large input penalty R ≻ 0. This follows from Theorem 11 by picking W(x) (Asm.9) as a Lyapunov function characterizing the open-loop (input-to state) stability.In case of hard state constraints, a large region of attraction requires a (proportional) increase in the prediction horizon N (Köhler, 2021, Thm. 4.12).
For minimum-phase systems, Köhler et al. (2022, Rk. 7) show asymptotic stability with a singular cost ℓ(x, u) = h(x) 2 Q and a very short horizon N (independent of γ).In the continuous-time case with an input-output stage cost ℓ Westenbroek et al. (2022, Thm. 2) show asymptotic stability for any horizon N by choosing a sufficiently small input penality R ≻ 0. Both results only hold locally in case of compact input constraints u ∈ U and a longer horizon is required to ensure a large region of attraction (Köhler, 2021, Thm. 4.50).

From analysis to design
In contrast to Sections 2-5, this section mainly provides analysis results and hence we next discuss how to apply these theoretical results.First, we explain how these theoretical results can be used to inform the choice of the prediction horizon N and stage cost ℓ.Then we mention some modifications to Problem (40) to enable theoretical guarantees with significantly shorter horizons N while keeping a simple design.
For a given stage cost ℓ, the previous results provide a lower bound N on the prediction horizon such that desired closed-loop properties can be guaranteed for N > N. Consider for example a trajectory tracking MPC scheme (Sec.3.3) with some feasible reference trajectory r(t) which is generated online (C.2).By verifying cost controllability (Asm.8) for any dynamic reference trajectory, we can derive a horizon N ensuring asymptotic stability for any reference trajectory (Köhler et al., 2019, Sec. III).
Depending on the system dynamics and the chosen stage cost ℓ, the derived horizon bound N may simply be too large for real-time implementation.For practical application, we may instead treat the prediction horizon N as given (due to computational requirements) and consider the stage cost ℓ as a design parameter.The qualitative results (Sec.6.2) provide some indication how the stage cost should be chosen and we mentioned special cases (open-loop stable, minimum-phase) for which we can derive arbitrarily small horizon bounds.Considering the horizon bounds (Sec.6.3.1), the stage cost ℓ should be chosen s.t. the cost controllability constant γ (Asm.8) is sufficiently small.Köhler et al. (2023, Sec. V) provide numerical examples to demonstrate how these theoretical results can be used to obtain a stage cost ℓ ensuring stability.Specifically, by parametrizing the stage cost ℓ in terms of few tuning variables (e.g. the input regularization), one can plot N over these tuning variables and thus directly map out feasible design choices.We emphasize that this analysis does not require any closed-loop simulation, but only numerical verification of Assumption 8/9, which can be done efficiently. 16 In Section 4, tracking MPC formulations use artificial setpoints to effectively deal with infeasible references and provide a large region of attraction, even for short horizons N.For the simple MPC formulation (Problem (40)), infeasible references result in an economic formulation and closed-loop properties require a potentially very large prediction horizon N.Even for feasible references, a large region of attraction requires a large prediction horizon N. Hence, it is desirable to merge the benefits of the simple design of Problem ( 40) and the properties of artificial references (Sec.4).Limón et al. (2018, Thm. 3) show that we can forgo the design of a terminal set constraint X f for setpoint tracking MPC formulations by using sublevel set arguments (cf.Sec.6.2.6).However, implementation still requires the design of a local CLF satisfying Condition (T.2) in Assumption 2. Soloperto et al. (2022, Thm. 2) directly integrate an artificial setpoint in Problem (40), without any terminal cost V f .The corresponding analysis characterizes the region of attraction as a sublevel sets of the tracking cost (cf.Sec.6.2.6) w.r.t.any artificial reference r ∈ Z r .This provides a large region of attraction and stability of the optimal setpoint with relatively short prediction horizons N, while the absence of terminal ingredients allows for a simple design.
Using a (local) CLF as a terminal cost V f can ensure stability properties for any horizon N ∈ I ≥1 (Sec.2).However, computing a local CLF can be cumbersome and avoiding this is one of the main motivations to consider the simple MPC formulation in Problem (40).In order merge the complementary advantages we investigate two questions: 1) Can we ensure stability with a shorter horizon N if we add a terminal cost V f satisfying a relaxed CLF condition?
2) Is the design of such a relaxed CLF significantly simpler?
1): Tuna et al. (2006, A 3) characterize a relaxed CLF as and similar conditions are used by Reble et al. (2012); Köhler and Allgöwer (2021); Köhler et al. (2023).For ǫ f = 0, Condition (47) corresponds to Assumption 2, however, (47) is also naturally satisfied for any positive definite terminal cost 16 Considering the examples by Köhler et al. (2023, Sec. V): For a linear system with n = 12 states, computing N for one dozen parameter combinations requires about one second.However, for a nonlinear system with n = 4 states, computing N for one parameter combination already takes about one minute.This increase in computation time for nonlinear systems results from approximating γ by simulating (open-loop) trajectories over a gridded set of initial conditions.
2): For practical application, we require a simple way to obtain a terminal cost V f .Beckenbach and Streif (2022) and Moreno-Mora et al. (2023) compute V f using approximate dynamic programming and relate ǫ f to the stopping condition.Grüne et al. (2010, Thm. 5.4) suggest V f (x) = ωℓ min (x) with a simple weighting ω ≥ 1, which is especially attractive if min u∈U ℓ min ( f (x, u)) ≤ ℓ min (x).In this case, asymptotic stability holds for any horizon N ∈ I ≥1 if ω is chosen sufficiently large (Köhler et al., 2023, Sec. VI), see also Reble et al. (2012) and Alamir (2018) regarding stability results by increasing the weighting over the horizon.Magni et al. (2001); Köhler and Allgöwer (2021) define V f (x) use a finite-horizon rollout, i.e., V f (x) = M−1 k=0 ℓ(x u (k, x), u k ) with the rollout horizon M ∈ I ≥1 and some locally stabilizing control law u ∈ U M .The resulting MPC formulation is equivalent to Problem (40) with a horizon N + M with an additional constraint on the last M inputs.Köhler and Allgöwer (2021, Prop. 4) provide an LP analysis to derive a tight constant ǫ f satisfying (47) and show that asymptotic stability can be ensured with a significantly smaller horizon N + M. Such a combination of online optimization and a finite-horizon rollout is also key in various reinforcement learning algorithms (Bertsekas, 2022).In the special case of linear systems with polytopic constraints, Raković and Zhang (2023, Thm. 4) show that this finite-horizon rollout can implicitly characterize the terminal set constraint X f and a scaled finite-horizon rollout cost provides a local CLF satisfying Assumption 2. 17 The implicit characterization of the terminal cost in terms of a finite-horizon rollout ensures that no redesign is required for online changing setpoints (cf.Sec.3.2/4.1)if a (locally) stabilizing feedback is known (Magni and Scattolini, 2005), which makes it very attractive for practical application.
For economic MPC without terminal ingredients, the theoretical bounds on the prediction horizon are rather qualitative.Zanon and Faulwasser (2018, Thm. 5) show that for any finite horizon N, the economic MPC scheme results in suboptimal operation.Zanon and Faulwasser (2018, Thm. 5) ensure (local) asymptotic stability by adding a linear gradient correction as a terminal cost.Liu and Liu (2016, Thm. 4) use a finite-horizon rollout of the economic cost ℓ e as a terminal cost, which ensures exponential stability for a sufficiently large rollout horizon M.

Discussion and Conclusions
In this section, we provide some high-level discussion and conclusions.First, we provide a concise summary of the different frameworks introduced in this article (Sec.7.1) and address some important extensions (Sec.7.2).Lastly, we discuss complementary advantages of these frameworks and possible unifying approaches (Sec.7.3).

Summary
In the following, we summarize the main tools and methods developed in Sections 3-6 and discuss how they address the challenges of dynamic operation (C.1)-(C.3).
In Section 3, we provide design procedures for the terminal ingredients V f , X f .For a given steady state, this design uses the linearization to construct a local CLF with the LQR (Chen and Allgöwer, 1998).For non-stationary operation (C.1), we use a local LTV model around a reference trajectory and construct a local CLF using a time-varying LQR (Faulwasser and Findeisen, 2011).To account for online changes in the reference (C.2), we develop a reference generic offline computation using tools from LPV systems to yield parametrized terminal ingredients for any feasible reference setpoint/trajectory (Köhler et al., 2020a).In addition, we discuss terminal equality constraints as a simple but conservative alternative (Sec.3.5.2).
In Section 4, we design tracking MPC formulations using artificial references.These designs address online changing operating conditions (C.2) by ensuring feasibility independent of online changes in the reference.Furthermore, the offset cost V o enables indirect specification of the optimal mode of operation through a (possibly infeasible) output target y d (C.3), while stability of the implicitly defined optimal steady state is ensured (Limón et al., 2018).To account for non-stationary operation (C.1), a periodic artificial reference trajectory is used and stability of the (unknown) optimal periodic trajectory is established (Limón et al., 2016;Köhler et al., 2020b).The deployment of these nonlinear tracking MPC schemes hinges on the design of terminal ingredients for online changing references (Sec.3).Important extensions include the design of an offset cost V o addressing non-convexity (Soloperto et al., 2022) and the reduction of the online computational demand by operating a tracking MPC and a planner at different frequencies (Köhler et al., 2020b).
In Section 5, we directly express the control goal in the cost ℓ e (C.3) using an economic MPC formulation (Faulwasser et al., 2018).The economic MPC scheme ensures that the closed-loop performance is no worse than tracking MPC schemes (Angeli et al., 2012).Non-stationary operation (C.1) in terms of periodic trajectories can also be directly addressed (Zanon et al., 2017).To account for online changing operating conditions (C.2), we use artificial setpoints (Sec.4).Optimality guarantees with economic stage costs require a selftuning weight β(t) (Müller et al., 2014a).We use artificial periodic references to account for non-stationary and online changing operating conditions (C.1)-(C.2),however, this may result in suboptimal performance.We recover the desired performance guarantees by introducing a shifted terminal cost Ṽf that yields a monotonic decrease condition (Köhler et al., 2020c).We also discuss stability and the design of terminal ingredients for economic MPC schemes.
In Section 6, we analyse MPC formulations without terminal ingredients.Given a suitable stabilizability condition, asymptotic stability is ensured with a sufficiently long horizon N and a tight bound for this horizon is computed using an LP analysis (Grüne et al., 2010).Since this framework requires no explicit offline design, the results can be directly applied to nonstationary operation (C.1) with online changing operating conditions (C.2).We address input-output/output references (C.3) assuming detectability/minimum-phase (Grimm et al., 2005;Köhler et al., 2022;Westenbroek et al., 2022) and a tight horizon bound N is obtained using an LP analysis (Köhler et al., 2023).The consideration of infeasible references complicates the analysis, however, (practical) stability of the optimal feasible trajectory can be ensured with a sufficiently large horizon (Köhler et al., 2019;Long and Xie, 2021).General economic stage cost can be handled similarly (Grüne, 2013;Grüne and Stieler, 2014;Grüne and Pirkelmann, 2020).Although most of these results consider global properties, we also study the region of attraction (Boccia et al., 2014), including corresponding bounds on the prediction horizon N (Köhler, 2021).By adding a terminal weight, stability can be ensured with a significantly shorter horizon N (Köhler et al., 2023).

Flexible time parametrization
In the following, we re-parametrize time-varying reference signals to increase the flexibility of the MPC approaches.
In Section 3.2, we introduce the trajectory tracking problem using a time-varying reference trajectory x r (t), t ∈ I ≥0 .This problem can be generalized to a path-following problem, where only the geometric curve of the reference is provided, i.e., x r (θ) with a scalar parametrization θ ∈ [0, 1].The timing can be flexibly changed by optimizing θ(t) > 0 as a free control input and a desired speed θ can be set in the cost function.Corresponding path-following MPC formulations with stability and convergence guarantees are derived by Faulwasser (2012); Faulwasser and Findeisen (2015); Yu et al. (2015), including an MPC formulation without terminal ingredients (Faulwasser et al., 2021).Such pathfollowing implementations are also successfully applied in different experiments (Faulwasser et al., 2016;Liniger et al., 2015;Romero et al., 2022).These approaches are also called model predictive contouring control (Lam et al., 2010).
It is natural to combine a path-following formulation with an artificial reference path (Sec.4).Sánchez et al. (2023) provide a first step in this direction, by optimizing a periodic artificial reference r ∈ R T and evaluating the offset cost V o w.r.t. a periodic reference curve.By jointly optimizing the path-progress θ, convergence to the optimal periodic curve is ensured.However, this approach still requires a fixed period length T in the artificial reference.Further research is required in this direction.
In summary, most of the designs in this article that use a timevarying reference can be generalized to use a more flexible time parametrization, which can significantly improve performance.

Large scale systems and distributed solutions
In the following, we discuss application of the presented approaches to large scale systems.Scalability for large systems is often achieved using distributed approaches, see the overviews by Christofides et al. (2013); Müller and Allgöwer (2017) on distributed MPC.We discuss design procedures that are scalable and preserve sparsity of the optimization problems.Hence, efficient distributed optimization methods can be applied (Engelmann et al., 2022), see also Köhler et al. (2019) regarding closed-loop properties under inexact distributed optimization.
Considering the design of terminal ingredients V f , k f (Sec.3), computing the CLF based on the LQR destroys the sparsity structure in Problem (4)/(10).Furthermore, the LMI based computation of parametrized terminal ingredients by Köhler et al. (2020a) does not scale to high dimensional systems.Conte et al. (2016) propose an LMI-based design of the terminal ingredients for linear systems using a distributed parametrization of V f , k f .This ensures that the offline design and the online optimization can be efficiently distributed.Similar distributed parametrizations can be utilized to compute parametrized terminal ingredients for nonlinear systems, cf.Wang et al. (2017).
In general, few existing results extend economic MPC (Sec.5) to large scale distributed systems (Müller and Allgöwer, 2017, Sec. 4.5).The design of the terminal ingredients for large scale systems also utilizes a distributed parametrization (Köhler, 2017, Sec. 2.2).Coordination of multiple agents with economic cost functions is addressed by Köhler et al. (2018b) using a separate coordination algorithm to compute reference setpoints.
MPC formulations without terminal ingredients (Sec.6) are simple to apply for large scale systems due to the absence of any offline designs.For example, Giselsson and Rantzer (2014) and Köhler et al. (2019, App.A) derive closed-loop guarantees for regulation and economic performance despite inexact distributed optimization.

Model mismatch and robustness
In this article, we assume that the system evolves exactly according to our model (1), which is rarely the case in practice.In general, recursive feasibility and closed-loop properties under model mismatch require a robust MPC design (Kouvaritakis and Cannon, 2016;Mayne, 2016).A simple robust MPC design uses a nominal prediction and confines the system state x(t) in a tube around this nominal prediction with an additional tube feedback, cf.Mayne et al. (2011); Mayne (2016); Raković et al. (2022); Sasfi et al. (2022); Kouvaritakis and Cannon (2016, Sec. 3.5).By using a nominal cost, the nominal closed-loop properties we derive in this article also apply to this nominal trajectory and the system state x(t) is confined to a neighbourhood of this nominal trajectory.
While the design of such tubes is beyond the scope of this article, we highlight a connection to the parametrized terminal cost V f (x, r) introduced in Section 3.4.Specifically, this design results in an incremental Lyapunov function (cf.Sec.3.5.4),which is used in the nonlinear robust MPC designs by Bayer et al. (2013); Singh et al. (2017); Köhler et al. (2018Köhler et al. ( , 2021)); Sasfi et al. (2022) to parametrize this tube.
In particular, asymptotically constant model mismatch can be compensated (asymptotically) using a disturbance observer or a velocity formulation (Muske and Badgwell, 2002;Pannocchia and Rawlings, 2003;Cisneros and Werner, 2020).This does not ensure convergence to an optimal steady state (21) in case of an economic stage cost ℓ e .Modifier adaptation estimates a correction for the steady-state optimization (21) that ensures convergence to the (economically) optimal steady state (Alamo et al., 2014;Faulwasser and Pannocchia, 2019;Vaccari et al., 2021).Developing similar approaches for more dynamic problems (C.1) is an open problem.

Complementary benefits and limitations
In the following, we contrast the complementary advantages of the different MPC frameworks presented in Sections 3-6 and highlight some unifying approaches.First, we analyse the role of artificial references (Sec.7.3.1).Then, we discuss benefits of using economic MPC formulations (Sec.7.3.2).Finally, we contrast the benefits and limitations of using terminal ingredients in MPC (Sec.7.3.3).

Artificial references
One of the key tools we use in Sections 4-5 to deal with online changing operating conditions (C.2) are artificial references (Sec.4).Artificial references ensure recursive feasibility and constraint satisfaction completely independent of the control goal, which is expressed by some target to be tracked or an economic cost, both of which may vary unpredictably online.The non-trivial part is that under nominal conditions, we obtain the same performance/stability guarantees that we would achieve with a standard MPC formulation with a fixed optimal references (cf.Thm.4/9).From a design perspective, the addition of an artificial reference does not require complex offline operations and is hence easy to apply.Additional benefits of artificial references include the large region of attraction and flexibility, e.g., if multiple setpoints are optimal (cf.Sec.4.2.3).Main challenges and limitations in the application of artificial references include: (i) design of terminal ingredients; (ii) periodicity requirement for dynamic references; and (iii) computational complexity.
(i) One drawback of this approach is arguably the fact that terminal ingredients need to be designed for all possible references, e.g., using Köhler et al. (2020a).However, even if a standard stabilizing MPC scheme (Problem (4)) is deployed, online changes in the reference require a mechanism to efficiently update the terminal ingredients online.The approach by Soloperto et al. (2022) circumvents this issue by using artificial references without any terminal ingredients.This combines many of the benefits of artificial references (Sec.4) and MPC without terminal ingredients (Sec.6), such as a simple design, a large region of attraction, and direct handling of online changing or infeasible references (C.2)-(C.3).
(ii) While achieving optimal steady-state operation with artificial setpoints r ∈ Z r is well understood, optimal operation with non-stationary targets (C.1) remains challenging.We address this issue using periodic artificial reference trajectories r ∈ Z T r and provide guarantees w.r.t.optimal periodic operation with period length T .This requires a priori knowledge of the optimal periodic length T , which can be relaxed using a suitable parametrization (Sec.7.2.1).More importantly, there are many control problems where optimal operation is completely non-periodic.For example, batch processes in process control or motion planning problems for autonomous robots are typically posed as finite-horizon control problems and searching for a periodic solution may be infeasible or highly suboptimal.
Addressing such problems with artificial references remains an open issue.
(iii) Considering the computational complexity, Problem ( 14)/(32) introduces an artificial setpoint, which requires r ∈ Z r ∈ R n+m additional decision variables and the steadystate constraints (16), which seems negligible for N ≫ 1. 18However, as noted in Section 4.4, the increase in computational complexity can be very large in case of artificial periodic reference trajectories r ∈ Z T r (cf.Problem (18)/( 35)), which introduce (n + m)T additional decision variables and T n additional nonlinear equality constraints.Köhler et al. (2020b) address this issue by decomposing Problem (18) into a tracking MPC and a planning problem (Sec. 4.4).This circumvents the computational increase by running a standard tracking MPC scheme while the planning problem is solved in parallel on a longer time-scale.However, multiple open issues remain, e.g., showing stability or considering economic formulations (Köhler, 2021, Rk. 3.89).More broadly, decomposing complex control problems into a standard MPC scheme combined with a more complex optimization problem solved on a slower timescale is a promising approach, cf., e.g., tracker-planner hierarchy (Schweidel et al., 2022) or asynchronous updates in robust MPC (Sieber et al., 2023).19

Directly minimizing economic costs
In Section 5, we discuss economic MPC formulations that directly optimize the performance measure ℓ e , e.g., reflecting production yield and energy consumption.This is in contrast to the tracking MPC formulations (Sec.2-4), which use a positive definite stage cost ℓ to regulate the system to some desired mode of operation.In the following, we discuss the benefits of directly using an economic stage cost ℓ e vs. a regulation cost ℓ.
A benefit of tracking MPC formulations is simplicity, i.e., we ensure stability of the optimal setpoint and hence closedloop operation is no worse than operation at this setpoint.In economic MPC, we may require additional modifications to ensure the same performance bounds with artificial references (Sec.5.3-5.4) and even more care is required for economic MPC formulations without terminal ingredients.The fact that economic MPC schemes do not necessarily guarantee stability may be a practical hindrance for deployment.For exam-ple, monitoring a process may be less trivial as significant fluctuations in the controlled variables could indicate some failure, even if these fluctuations do improve the economic performance.
Theorem 5 ensures that the average performance of an economic MPC scheme is no worse than any tracking MPC scheme.There are numerous examples demonstrating significant performance benefits using economic MPC schemes (Rawlings et al., 2008(Rawlings et al., , 2012;;Ellis et al., 2017).An economic MPC can strictly outperform steady-state operation on average (cf.(Faulwasser et al., 2018, Sec. 3.4)).Even if the system is optimally operated at the steady state, the transient performance of economic MPC formulations is typically superior (Rawlings et al., 2008(Rawlings et al., , 2012)).This is particularly relevant if such transient phases occur regularly due to persistent disturbances or frequent changes in the operating conditions.
The adoption of economic MPC is especially relevant in applications where the objective is largely independent of the stability of the system, e.g., in HVAC systems economic cost functions are common to reduce energy consumption (Taheri et al., 2022, Table 3).Even though similar economic considerations are paramount in various industries, the deployment of tracking MPC formulations remains the norm.
The introduction of economic MPC schemes in practice may be facilitated by formulations that also enable a user to flexibly set desired stability properties.Approaches to address this issue include Lyapunov constraints (Heidarinejad et al., 2012;Ellis et al., 2017), average constraints (Müller et al., 2014b,c), multi-objective formulations (Soloperto et al., 2020;Eichfelder et al., 2022), or using the external variable y e to flexibly change the cost online (cf.Köhler et al. (2020c, Fig. 4), Sec.5.3).

On terminal ingredients in MPC
In the following, we discuss advantages and limitations of using terminal ingredients in MPC, see also the discussions by Mayne (2013) and Grüne and Pannek (2017, Sec. 7.4).First, we provide a general discussion contrasting the standard MPC formulation (Sec.2) to MPC formulations without terminal ingredients (Sec.6).Then, we focus on the challenges related to dynamic operation (C.1)-(C.3).Finally, we highlight some approaches that combine ideas from both frameworks.
Considering the closed-loop performance and region of attraction, QINF is generally superior to TEC, see, e.g., comparisons by Chen and Allgöwer (1998, Sec.5), Raff et al. (2006, Sec.V), Köhler et al. (2020b, Sec. 4.1), andKöhler et al. (2020a, Sec. 5.2).General statements regarding the performance of UC vs. QINF/TEC are difficult, however, two important extreme cases can be regarded.(i) Suppose the terminal set constraint is inactive and the terminal cost is a close approximation of the infinite-horizon optimal cost.Then, the performance of QINF is close to optimal (cf.Grüne and Pannek (2017, Thm. 5.22), Köhler (2021, App.A)), while the performance of UCON with a short horizons N can be highly suboptimal.(ii) Consider an open-loop stable systems with no state constraints: Theorem 10 can ensure any desired performance bound α N ∈ [0, 1) w.r.t.optimal operation (44) for UCON with a sufficiently large (finite) horizon N. On the other hand, QINF/TEC typically has a bounded feasible set, which yields a smaller region of attraction and may result in severe performance limitations.Intuitively, we can judge the effect of a terminal cost V f and terminal set X f by how close they approximate the infinite-horizon optimal cost J ⋆ ∞ : QINF provides a very good approximation close to the origin, however, the bounded terminal set X f can provide an arbitrarily bad approximation for feasible states outside of the terminal set.
UCON and QINF can be applied if the linearization is stabilizable, while TEC requires a local controllability condition.Considering the design complexity: both TEC and UCON require no offline design, while QINF requires an LQR design based on the linearization and the determination of a suitable terminal set scaling α > 0. Given terminal ingredients V f , X f (Asm.2), we can directly compute constants γ, c satisfying cost controllability (Asm.10), cf.(Schulze Darup and Cannon, 2015).Similarly, given constants γ, c satisfying the cost controllability condition (Asm.10), we know that there exist terminal ingredients satisfying Assumption 2. However, we do not have any analytical formula for the local CLF V f nor is there any reason to assume that a quadratic function V f is valid outside of a small set of initial conditions.Hence, the design of a CLF V f and a positive invariant set X f for QINF is intrinsically more complex.
In the following, we focus on challenges related to dynamic operation (C.1)-(C.3).The design of a terminal cost V f for time-varying references (C.1) or online changing references (C.2) becomes significantly more complex (Sec.3).In addition, evaluation of the terminal cost/set V f /X f in Problem (10) requires the state reference x r (t), which is difficult to implement if only an output reference y r (t) is available (C.3).Furthermore, all theoretical properties break down if the provided reference does not satisfy the constraints or dynamics (C.3).Although the deployment of artificial references can avoid some of these issues, this results in additional challenges (cf.Sec.7.3.1).In contrast, MPC formulations without terminal ingredients can be directly applied to time-varying or online changing references (C.1)-(C.2).Even output references y r (t) or infeasible references can be handled (cf.Sec.6.2.5).Hence, the advantages of a simple MPC formulation without terminal ingredients become more pronounced when considering such dynamic operation.However, derivation of a sufficiently long horizon N also becomes more challenging and in general a long prediction horizon N may be needed due to the absence of terminal ingredients.Such MPC formulations without terminal ingredients are also particularly popular in robotics experiments with dynamic operation, cf., e.g., Liniger et al. (2015); Faulwasser et al. (2016);Rosenfelder et al. (2023); Romero et al. (2022); Rickenbach et al. (2023).This important factor is often neglected in discussions regarding benefits and drawbacks of terminal ingredients in MPC, cf.Mayne (2013) and Grüne and Pannek (2017, Sec. 7.4).One important exception is economic MPC (Sec.5): There exist results for time-varying economically optimal operation without terminal ingredients (Köhler et al., 2019;Grüne and Pirkelmann, 2020;Long and Xie, 2021), however, the corresponding conditions are very challenging to verify.On the other hand, there exist simple design methods for economic MPC schemes with terminal ingredients that provide strong theoretical properties.
Lastly, we discuss approaches that unify the benefits of MPC frameworks with and without terminal ingredients.Limón et al. (2006) show that we can drop the terminal set constraint X f in Problem (4) for any initial condition in a region of attraction x ∈ X V .Limón et al. (2018, Sec. III.B) combine this with artificial setpoints to handle infeasible references.Soloperto et al. (2022) further remove the requirement of a local CLF V f , thus simplifying the design.Magni et al. (2001) and Köhler and Allgöwer (2021) use a finite-horizon rollout cost V f to approximate a CLF.Köhler et al. (2023) provide a theoretical analysis to ensure stability with a short horizon N using such an approximate CLF, see also the continuous-time results by Reble et al. (2012).For linear systems with polytopic constraints, Raković and Zhang (2023) provide an implicit characterization of V f and X f satisfying Assumption 2 using a finite-horizon rollout.These finite-horizon rollouts provide benefits comparable to a well-designed terminal cost V f without the associated offline design requirements.In combination, these approaches can be implemented with a short horizon N, without any complex offline design, and for infeasible references.