On the Undesired Equilibria Induced by Control Barrier Function Based Quadratic Programs

In this paper, we analyze the system behavior for general nonlinear control-affine systems when a control barrier function-induced quadratic program-based controller is employed for feedback. In particular, we characterize the existence and locations of possible equilibrium points of the closed-loop system and also provide analytical results on how design parameters affect them. Based on this analysis, a simple modification on the existing quadratic program-based controller is provided, which, without any assumptions other than those taken in the original program, inherits the safety set forward invariance property, and further guarantees the complete elimination of undesired equilibrium points in the interior of the safety set as well as one type of boundary equilibrium points, and local asymptotic stability of the origin. Numerical examples are given alongside the theoretical discussions.


I. INTRODUCTION
System safety has recently increasingly gained attention in control community. One formal definition regarding system safety relates to a set of states, referring to as the safety set, that the system is supposed to evolve within. The study of control barrier functions (CBFs) [1]- [3] enforces the safety set to be forward invariant and asymptotically stable by requiring a point-wise condition on the control input. A similar pointwise condition was earlier studied [4] under the concept of control Lyapunov functions (CLFs), where system stability is concerned. In [1], a CLF-CBF based quadratic program (CLF-CBF-QP) formulation is proposed with an intention to provide a modular, safe, and stabilizing control design. Thanks to the increasing computational capabilities in modern control systems and its modularity design nature, the CLF-CBF-QP formulation has been applied successfully to a wide range applications, e.g., in adaptive cruise control [1], bipedal robot walking [5], multi-robot coordination, verification and control [6]- [8].
However, one major limitation with the CLF-CBF-QP formulation is that, while the controller ensures system safety, no formal guarantee has been achieved on the system trajectories converging to the origin (the unique minimum of the CLF). This is mainly due to the relaxation on the CLF constraint in the program for the sake of its feasibility. In fact, [9] shows that even for a single integrator dynamics with a circular obstacle, the program could induce undesired equilibrium points that are locally stable. There are many attempts in the literature trying to achieve a safe and (locally) stabilizing control law by modifying the original formulation and/or posing additional assumptions. Local asymptotic stability is proved in [10] with a modified quadratic program and, additionally, assuming that the CBF constraint is inactive around the origin; yet how to fulfill such an assumption has not been discussed. In [11], an approximate dynamic program framework is proposed that embeds the cost of violating the CBF constraint in an optimal value function over infinite time horizon; yet no formal guarantee can be asserted through the adaptive approximation of the oracle optimal value function whose existence is merely assumed. [9] examines the equilibrium points of the closed-loop system, and introduces an extra CBF constraint into the original QP which removes boundary equilibria in the original QP formulation; yet the assumptions on the control coefficient matrix to be full rank and the feasibility of the modified QP hinder its applications. In [12], the compatibility between the CLF and the CBF is discussed, and a sufficient condition on the regions of attraction is proposed. The condition is however conservative and checking such conditions for general nonlinear systems remains challenging. In our previous work [3], we guarantee that, for general control-affine systems, by modifying a CBF candidate, the nominal control law, which can be derived from a CLF, can be implemented without any modification in an a priori given region inside the safety set, and thus ensuring local stability follows. Yet it still cannot rule out the existence of undesired equilibria inside the safety set.
In this paper we present a new control barrier function-based quadratic program. We show that without any assumptions other than those taken in the original program [1], the proposed formulation simultaneously guarantees the forward invariance of the safety set, the complete elimination of undesired equilibrium points inside the safety set, and the local asymptotic stability of the origin. Before proving this main result, we revisit the original CLF-CBF-QP formulation and characterize all possible equilibria of the closed-loop system. While similar results have been partially reported before, here we make an effort to remove assumptions that the control coefficient matrix is full rank or the CBF is of relative degree one as done in previous works. We further show how to choose the parameter in the original QP formulation and its impact on the closed-loop system equilibria. Our main result, a new quadratic program formulation, then follows from the previous analysis.
II. PRELIMINARY Notation: The operator ∇ : C 1 (R n ) → R n is defined as the gradient ∂ ∂x of a scalar-valued differentiable function with respect to x. The Lie derivatives of a function h(x) for the systemẋ = f(x) + g(x)u are denoted by L f h = ∇h f(x) ∈ R and L g h = ∇h g(x) ∈ R 1×m , respectively. The interior and boundary of a set A are denoted Int(A ) and ∂A , respectively. A continuous function α : [0, a) → [0, ∞) for a ∈ R >0 is a class K function if it is strictly increasing and α(0) = 0 [13].
Consider the nonlinear control affine systeṁ where the state x ∈ R n , and the control input u ∈ R m . We will consider the simplified case where f(x), g(x) and the controller u(x) are locally Lipschitz functions in x 1 . Denote by x(t, x 0 ) the solution of (1) starting from x(t 0 ) = x 0 . By standard ODE theory [14], there exists a maximal time interval of existence I(x 0 ) and x(t, x 0 ) is the unique solution to the differential equation (1) for all t ∈ I(x 0 ), x 0 ∈ R n . A set A ⊂ R n is called forward invariant, if for any initial condition is an extended class K function if it is strictly increasing and α(0) = 0.
Note that the extended class K functions addressed in this paper will be defined for a, b = ∞.
Definition 2 (CLF). A positive definite function V : R n → R is a control Lyapunov function (CLF) for system (1) if it satisfies: where γ : R ≥0 → R ≥0 is a class K function.
Consider the safety set C defined as a superlevel set of a differentiable function h : R n → R: Definition 3 (CBF). Let set C be defined by (3). h(x) is a control barrier function (CBF) for system (1) if there exists a locally Lipschitz extended class K function α such that: In [1], the CBF h(x) is defined over an open set D containing the safety set C . Here we instead require the CBF condition to hold in R n for notational simplicity. All the results in this paper remain intact even when h(x) is defined only over an open set D, except that a set intersection operation with D is needed for all the sets of states in the following derivations. Without loss of generality, the desired equilibrium point in (1) is assumed to be the origin and to lie inside the set C .

A. Quadratic Program Formulation
The minimum-norm controller proposed in [1] is given by the following quadratic program with a positive scalar p: which softens the stabilization objective via the slack variable δ, and thus maintains the feasibility of the QP, i.e., if h(x) is a CBF, then the quadratic program in (5) is always feasible. A controller u(x) given by the quadratic program satisfies the CBF constraint for all x ∈ R n , thus the safety set C is forward invariant using Brezis' version of Nagumo's Theorem [3]. However, due to the relaxation in the CLF constraint, the stabilization of the system (1) is generally not guaranteed.

III. CLOSED-LOOP SYSTEM BEHAVIOR
In this section, we investigate the point-wise solution to the quadratic program in (5), the characterization of equilibrium points of the closed-loop system, and the choice of the QP parameter p in (5). Hereafter we denote the control input given as a solution of (5) as u (x) and the closed-loop vector field Note that although similar results in Sections III-A and III-B have also been partially reported in [1] and [9], here we aim at giving further technical details and insights. One notable difference is that we do not assume g is full rank as in [9] nor L g h = 0, ∀x ∈ R n as in [1].
A. Explicit solution to the quadratic program Theorem 1. The solution to the quadratic program in (5) is given by where the set Ω clf cbf is given in (17), Before diving into the proof, we note that for the domain sets in (6), a bar being in place refers to the inactivity of the corresponding constraint. The subscript cbf, 1 refers to the case when the CBF constraint is active and L g h = 0, while cbf, 2 refers to the case when the CBF constraint is active and Proof. The Lagrangian associated to the QP (5) is Here λ 1 ≥ 0 and λ 2 ≥ 0 are the Lagrangian multipliers. The Karush-Kuhn-Tucker (KKT) conditions are then: Case 1: Both the CLF and CBF constraints are inactive.
In this case, we have From (9), δ = λ 1 /p = 0. From (8) and λ 1 = λ 2 = 0, To find out the domain where this case holds, substituting (16) into (12) and (13), and further noting that δ = 0, we obtain Case 2: The CLF constraint is inactive and the CBF constraint is active. In this case, we have From (9), δ = λ 1 /p = 0. We consider the following two sub-cases. 1) L g h = 0. Note that λ 1 = 0, L g h = 0, then from (8), λ 2 could be any positive scalar. To obtain the domain where this case holds, substituting (22) to (18) and (19) and noting that δ = 0, we obtain To find out the domain where this case holds, substituting (24) into (18) and noting that δ = 0, we obtain that the CLF constraint being inactive implies Case 3: The CLF constraint is active and the CBF constraint is inactive.
In this case, we have From (8) and (29), we obtain u+λ Thus we get In the domain where this case holds, Case 4: Both the CLF constraint and the CBF constraint are active.
In this case, we have From (8), (9), we obtain u = −λ 1 L g V + λ 2 L g h and δ = λ 1 /p. Substituting u and δ into (33) , (34), we obtain , and x 2 y 2 ≥ (x y) 2 , ∀x, y ∈ R n , we know that ∆ = 0 if and only if L g h = 0 for any p > 0. We discuss the solution to (37) in the following two sub-cases.
In this case, ∆ = 0. From (37), we know λ 2 could be any positive scalar, and F h = 0. Furthermore, in view of (8), we obtain and, in view of (9), δ = In this subcase, we assumed that both the CLF and the CBF constraints are active and L g h = 0, which implies in view of (38). In view of (37) and L g h = 0, we obtain F h = 0. Thus the domain where this subcase holds is given by In this case, Thus, λ 1 and λ 2 are given by . (42) From (8), we obtain with λ 1 in (41) and λ 2 in (42). In the domain where this case holds, λ 1 ≥ 0, λ 2 ≥ 0, and L g h = 0, and it implies

B. Existence of equilibrium points
It is known that the quadratic program in (5) will induce undesired equilibria for the closed-loop system [9]. Here we revisit this problem without assuming g is full rank as in [9] nor L g h = 0, ∀x ∈ R n as in [1].
Theorem 2. The set of equilibrium points of the systemẋ = f(x) + g(x)u (x) with the controller u resulting from (5) is with λ 1 given in (41) and λ 2 given in (42).
Proof. We first show the following facts.
Fact 1: No equilibrium points exist when the CLF constraint in (5) is inactive. Consider the case when the CLF constraint is inactive, meaning F V + L g V u − δ < 0 and λ 1 = 0. The equilibrium condition f cl (x) = 0 implies that L f cl V = 0, thus is positive definite. This is an expected conclusion since no equilibrium points at whichV (x) < 0 exist.

Fact 2:
No equilibrium point exists in R n \ C .
At an equilibrium point x eq , the CBF constraint is simplified as α(h) ≥ 0, implying that the point does not lie outside of the set C . This is also quite intuitive because the integral curves starting from any states outside the set C will asymptotically approach the set C so no equilibrium points exist there.
Fact 3: Consider an equilibrium point x eq . Then x eq ∈ ∂C if and only if the CBF constraint is active at that point.
Sufficiency: In view that the CBF constraint is active at x eq , we have L f h(x eq ) + L g h(x eq )u + α(h(x eq )) = L f cl h(x eq ) + α(h(x eq )) = 0. Note that x eq is an equilibrium point, i.e, f cl (x eq ) = 0, thus α(h(x eq )) = 0, which implies x eq ∈ ∂C . Necessity: Since x eq is an equilibrium point and x eq ∈ ∂C , i.e., h(x eq ) = 0, we obtain L f h(x eq ) + L g h(x eq )u + α(h(x eq )) = L f cl h(x eq )+α(h(x eq )) = 0, i.e., the CBF constraint is active.
From Fact 1, we know that the the equilibrium points can only exist when the CLF constraint is active, i.e., in the sets Ω clf cbf , Ω clf cbf, 1 and Ω clf cbf,2 . Furthermore, the equilibrium points need to satisfy In the following we will discuss these three cases.
From Fact 2, we know that the equilibrium points can only be on the boundary or inside the set C . From Fact 3, equilibrium points lying on ∂C implies that the CBF constraint is active, thus the equilibrium points in this case lie inside the set C , as given in (45).

C. Choice of QP parameter
In this subsection, we discuss the choice of different p's in (5) and its impact on the closed-loop equilibrium points. The motivation is to remove undesired equilibrium points as much as possible or to confine them within a small region around the desired equilibrium (the origin). Due to space limitations, we only discuss the equilibrium points in Int(C ). A similar analysis can be carried out for equilibria on the boundary of the safety set.
From Theorem 2, all the equilibrium points in Int(C ) are in E clf cbf , where the following holds Note that for a given system in (1), a given CLF V (x) and a given class K function γ(·), f, g, γ(V ) and L g V are functions of the state x. We propose the following two propositions on choosing p. Proposition 1. If there exists a positive constant p such that no point in the set Ω clf cbf ∩ Int(C ) except the origin satisfies (52), then, with such a p in (5) applied, no equilibrium points except the origin exist in Int(C ).
The proof is evident in view of Theorem 2 and Corollary 1 and thus omitted here. Two numerical examples are given below.
Example 1. Consider the following system with the system state x = (x 1 , x 2 ) , a given CLF satisfies the CLF condition in Definition 2. From (52), by left multiplying ∇V on both sides, one (c) p = 100. Fig. 1: Comparison of the system trajectories in Example 1 with varying p values. The obstacle region is in dark green. All the simulated system trajectories converge to the origin, except one which converges to an equilibrium point on the boundary of the safety set.
Let p be any positive scalar. Then (54) does not hold for any x ∈ R 2 except the origin. Thus, no equilibrium points except the origin exist inside the set C , no matter what CBF h(x) is chosen. In Fig. 1, the obstacle region (in dark green) is {x ∈ R 2 : x − (0, 4) ≤ 2} and the CBF is given by h(x) = x − (0, 4) 2 −4 and α(x) = x, ∀x ∈ R. We observe that all the simulated trajectories converge to the origin, except one that converges to an equilibrium point on the boundary of the safety set.
Example 2. Consider the following system with the system state x = (x 1 , x 2 ) , a given CLF V (x) = satisfies the CLF condition in Definition 2. To show h(x) is a CBF, we only need to examine whether or not L f h(x) + α(h(x)) ≥ 0 when L g h(x) = −0.15x 1 − 0.2x 2 = 0 (otherwise, with a non-zero coefficient, we can always find a u that satisfies the CBF condition in Definition 3). Substituting From the first row, we obtain x 2 = 0. Substituting x 2 = 0 into the second row, we have x 1 = 15p 112 x 3 1 . Thus, x 1 = 0, ± 112/15p, p > 0. Proposition 1 dictates x = (x 1 , x 2 ) ∈ Int(C ), and recall that C is the superlevel set of the CBF h(x). Thus, we conclude that for 0 < p < 16/105 ≈ 0.152, there exists only one equilibrium point (the origin) in Int(C ), and for p > 16/105, there exist three equilibrium points in Int(C ). This conclusion is verified by the simulation results in Fig. 2. (a) p = 0.1.
(c) p = 10. Fig. 2: Comparison of the system trajectories in Example 2 with varying p values. The obstacle region is in dark green. When p = 0.1, the system trajectories converge to two undesired equilibrium points on the boundary of the safety set. When p = 1 or 10, the system trajectories instead converge to equilibrium points inside the safety set.
This example is of interest because: 1) here neither g is full rank nor L g h = 0, ∀x ∈ R n , which is required in previous works; 2) it demonstrates that, under the QP formulation in (5), the existence of undesired equilibria inside the safety set can depend on the value of p.
Determining a p that satisfies the assumptions in Proposition 1 could be difficult for general nonlinear systems. One systematic way to comply with these assumptions is given in Section IV with a new quadratic program formulation. Alternatively, we could tune p to adjust the positions of equilibrium points inside the set C , albeit with mild additional assumptions, as given in the following proposition.
Proposition 2. Assume that L g V L g V = 0, ∀x ∈ R n \ {0}, and there exists a class K ∞ function γ 1 such that γ 1 ( x ) ≤ V (x). Ifv := sup x∈R n \{0} L f V LgV LgV exists, then all the possible equilibrium points x eq inside the set C are bounded by Proof. From (52), pγ(V )L g V L g V = L f V , and we further obtain Note thatv = sup x∈R n \{0} L f V LgV LgV . Thus, all the possible equilibrium points inside the set C are bounded by (57).
Proposition 2 implies that we can confine the equilibrium points inside the set C arbitrarily close to the origin by choosing a greater p. A numerical example is given below.
Example 3. Consider the following system with the system state x = (x 1 , x 2 ) , a given CLF V (x) = ) satisfies the CLF condition in Definition 2.
Note that (c) p = 100. Fig. 3: Comparison of the system trajectories in Example 3 with varying p values. The obstacle region is in dark green. All of the simulated trajectories except one converge to a neighborhood region of the origin, which shrinks as p becomes larger.
sup x∈R 2 \{0} L f V LgV LgV = sup x∈R 2 \{0} 1 = 1. Thus, all possible equilibrium points x eq inside the set C are bounded by x eq ≤ 2/p. In Fig. 3, the obstacle region (in dark green) is {x ∈ R 2 : x−(0, 4) ≤ 2}, and the CBF is given by h(x) = x − (0, 4) 2 −4 and α(x) = x, ∀x ∈ R. We observe that all of the simulated trajectories except one converge to the neighborhood region of the origin, the size of which depends on the parameter p.

IV. A NEW QP-BASED CONTROL FORMULATION
In this section, we propose a new CLF-CBF based control formulation that simultaneously guarantees the forward invariance of the safety set C , the elimination of undesired equilibrium points inside the safety set C , and the local asymptotic stability to the origin.
Consider the nonlinear control affine system in (1) with a control Lyapunov function(CLF) V and a control barrier function(CBF) h. The new control formulation is given as follows. Let a nominal controller u nom : R n → R m be locally Lipschitz continuous. Rewrite (1) aṡ where f (x) := f(x) + g(x)u nom (x), u (x) := u(x) − u nom (x). In the following we will solve a new quadratic program to derive the virtual control input u (x) and the actual control input is then obtained by The virtual control input u is calculated by the following quadratic program with a positive scalar p: Theorem 3. Consider the nonlinear control affine system in (1) with a control Lyapunov function(CLF) V and a control barrier function(CBF) h with its associated safety set C . If the nominal control u nom satisfies the CLF condition (2), and the control input in (61) is applied to (1), then 1) the set C is forward invariant; 2) no equilibrium points except the origin exist in Int(C ); 3) the origin is locally asymptotically stable.
Proof. Consider the transformed systemẋ = f (x) + g(x)u (x). Since h is a CBF for the original system in (1), i.e., ∀x ∈ R n , ∃u ∈ R m such that Thus h is also a CBF for the transformed system in (60). It further indicates that the quadratic program in (62) is feasible for all x ∈ R n . V is also a valid CLF for the transformed system since the CLF condition in (2) is fulfilled with u = 0. Using Brezis' version of Nagumo's Theorem [3], we further obtain that the resulting u will render the safety set forward invariant.
Assume that there exists an equilibrium point x eq , x eq = 0 inside the set C . From Theorem 2, we know that f (x eq ) = pγ(V (x eq ))g(x eq )L g V (x eq ).
By left multiplying ∇V on both sides, we further obtain that L f V (x eq ) = pγ(V (x eq ))L g V (x eq )L g V (x eq ). For any positive number p and any x eq = 0, on the right-hand side, we know γ(V (x eq )) > 0, L g V (x eq )L g V (x eq ) ≥ 0. Since u nom (x) satisfies the CLF condition, we obtain Thus it yields a contradiction, implying no undesired equilibrium points exist inside the safety set C from Proposition 1.
Since u nom (x) satisfies the CLF condition, we have f (0) = 0, F V := L f V +γ(V ) ≤ 0 for all x ∈ R n . Note that F h (0) = L f h(0) + α(h(0)) = α(h(0)) > 0. By continuity, we know that there exists an > 0 such that for all x ∈ B := {x ∈ R n : x ≤ }, F h (x) > 0. Applying Theorem 1 with respect to the quadratic program in (62), we next show that for all x ∈ B , the optimal solution is δ (x) = 0. This fact is obtained by examining δ(x) in every domain and keeping in mind that 1) if F V (x) < 0, then x ∈ Ω clf cbf and δ(x) = 0; 2) if F V (x) = 0, then x lies in Ω clf cbf ∪ Ω clf cbf ∪ Ω clf cbf,1 , and δ(x) = λ 1 (x)/p = 0 by examining their respective λ 1 (x)s. We further obtain that With a standard Lyapunov argument [13], we then deduce that the origin is locally asymptotically stable. Remark 1. Any locally Lipschitz u nom : R n → R m that renders L f V + L gunom V negative definite and satisfies f(0) + gu nom (0) = 0 is a valid nominal controller in the new QP formulation in (62) and the three properties in Theorem 3 still hold for this u nom . The proof can be carried out in a similar manner.

Remark 2.
A nominal controller and a quadratic program with only the CBF constraint has been considered in [15, Equation (CBF-QP)]. We instead here still have a CLF constraint in the QP. This constraint may seem redundant at a first glance, but it helps removing the undesired equilibria inside the safety set and aligning the resulting controller to a stabilizing controller.  It is tempting to claim from Theorem 3 that the resulting controller guarantees that all integral curves converge to origin. Yet in general this is not true because 1) the integral curves may converge to the equilibrium points on ∂C ; 2) limit cycles, or other types of attractors may exist in the closed-loop system. Actually, for the scenario in Fig. 1, global convergence with a smooth vector field is impossible due to topological obstruction [16].
Example 4. Consider the system in (59). As shown in Fig. 3, if the original QP in (5) is applied, then all of the simulated system trajectories except one converge to a neighborhood region of the origin, the size of which is determined by p.
If the new control formulation in (61) is applied, and we choose u nom = −2x, then the transformed system is given in (53). From Fig. 1, we observe that all of the simulated system trajectories except one converge to the origin, and not merely to a neighborhood of it, no matter what value of p is chosen.
Example 5. Consider the system in (55). As shown in Fig. 2, if the original QP in (5) is applied, then the simulated system trajectories converge to certain undesired equilibria instead of the origin. If the new control formulation in (61) is applied, and we choose u nom = −2x 1 − x 2 , then the transformed system is With the same CLF and CBF functions as in Example 2, we obtain the numerical results shown in Fig. 4. We observe that all the simulated system trajectories converge to the origin, not merely to a neighborhood of it, no matter what value of p is chosen.
The proposed new QP formulation is favorable in many regards. Assumption-wise, what it requires is the same as that of the original quadratic program (5): the existence of a CLF and a CBF for the nonlinear system (1). Computation-wise, this new formulation does not add extra computations since the CLF-induced u nom can be obtained in an analytical form [4]. Finally, the proposed formulation provides simultaneously theoretical guarantees on both system safety and stability.

V. CONCLUSION
In this paper, we have derived, for general control-affine systems, point-wise analytical solutions to the widely used CLF-CBF based quadratic program and characterized all possible closed-loop equilibrium points. We further provide analytical results on how the parameter in the program should be chosen to remove the undesired equilibrium points or to confine them in a small neighborhood of the origin. Our main result, a new quadratic program formulation, is then presented. Without any assumptions other than those taken in the original program, the proposed formulation guarantees simultaneously for the first time forward invariance of the safety set, complete elimination of undesired equilibrium points inside it, and local asymptotic stability of the origin.