Elsevier

Automatica

Volume 69, July 2016, Pages 101-116
Automatica

Constrained distributed optimization: A population dynamics approach

https://doi.org/10.1016/j.automatica.2016.02.004Get rights and content

Abstract

Large-scale network systems involve a large number of states, which makes the design of real-time controllers a challenging task. A distributed controller design allows to reduce computational requirements since tasks are divided into different systems, allowing real-time processing. This paper proposes a novel methodology for solving constrained optimization problems in a distributed way inspired by population dynamics. This methodology consists of an extension of a population dynamics equation and the introduction of a mass dynamics equation. The proposed methodology divides the problem into smaller sub-problems, whose feasible regions vary over time achieving an agreement to solve the global problem. The methodology also guarantees attraction to the feasible region and allows to have few changes in the decision-making design when a network suffers the addition/removal of nodes/edges. Then, distributed controllers are designed with the proposed methodology and applied to the large-scale Barcelona Drinking Water Network (BDWN). Some simulations are presented and discussed in order to illustrate the control performance.

Introduction

An approach to design control systems is to express the desired performance of the plant as an optimization problem with multiple constraints, e.g., minimization of the error, minimization of the norm of states, minimization of the energy associated to control actions, all of those objectives subject to physical and/or operational constraints. When the system involves a large number of states, the design of optimization-based controllers becomes challenging, because of the lack of centralized information or because of other implications associated to information (e.g., communication issues, costs, reliability). The limitation regarding information availability demands the development of distributed optimization techniques that achieve an optimal point of a performance cost function for the total system by using only local and partial information. There are many distributed optimization applications in engineering, and most of them using a network systems approach (Bertsekas, 2012, Gao and Cheng, 2005, Simonetto et al., 2010, Simonetto et al., 2011). These problems have been solved by using distributed optimization algorithms based on the Newton method (Jadbabaie et al., 2009, Wei et al., 2013), the sub-gradient method (Johansson et al., 2008, Zhu and Martinez, 2012), and the consensus protocol (Johansson et al., 2008, Notarstefano and Bullo, 2011, Zhang and Liu, 2014), among other techniques. On the other hand, game theory studies the interaction of decision makers and the interconnection of decision making elements based on local information. From this perspective, game-theoretical tools become very useful to describe the behavior of distributed engineered systems (Marden & Shamma, 2015). One important characteristic of this theoretical approach is the Nash equilibrium concept, which describes how a global objective is reached based only on local decisions. The task to reach a global objective with partial information is one of the main aspects in distributed optimization problems. This problem may be seen as a multi-agent case in which there are local interactions among them. Furthermore, evolutionary game theory describes the previously mentioned model of agents interacting but also considering a determined population structure, i.e., constraints in the interaction among agents, (Nowak, 2006). From this point of view, this theory is suitable to design intelligent systems and controllers for systems where there are local decision makers (local controllers) and achieving a global performance and/or global goal under a specific structure, which is given by the topology of the system (e.g., energy networks, water networks, transportation networks, etc.). Also, game theory has become an important and powerful tool for solving optimization problems since the Nash equilibrium corresponds to the extreme of a potential function satisfying the Karush–Kuhn–Tucker (KKT) first order condition (Sandholm, 2010). This property is commonly used in a class of games known as potential games, which have gotten special importance in the solution of engineering problems. For instance, in Marden (2012) potential games are widely studied from the perspective of state-based games. Furthermore, some kind of optimization problems can be solved by finding a Nash equilibrium for an appropriate designed game, and the consideration of only local information allows to solve distributed optimization problems (Arslan & Shamma, 2004). For instance, in Gharesifard and Cortes (2013) a distributed convergence to Nash equilibria in two networks is discussed for zero-sum games. In Pantoja and Quijano (2011), distributed optimization has been applied using replicator dynamics (one of the six fundamental population dynamics), based on local information. In Li and Marden (2014), the design of utility functions for each agent in order to decouple constraints is presented, and the usage of penalty functions and barrier functions is discussed. The design of local control laws for individual agents to achieve a global objective is proposed in Li and Marden (2013), which has been extended in Zhang, Qi, and Zhao (2013) by using matrix theory. The consideration of dynamics in the system-equivalent graph that describes information sharing among decision variables is paramount since some network systems in engineering might grow (e.g., drainage network systems, drinking water networks, distributed generation systems). These dynamics represent an addition or removal of elements to/from the network. Moreover, the connectivity of the network elements could change over time (e.g., re-configuration systems), which could affect availability of information. In Li and Marden (2012), variations on the graph that determines the system information sharing are studied, where the set of communication links varies with a certain probability.

The main contribution of this paper is to introduce a novel methodology to solve constrained optimization problems in a distributed way, inspired by the population dynamics studied in Sandholm (2010). Different from the already published population dynamics approaches, this method adds dynamics to the population masses, making the population simplex vary properly over time making the method robust (Barreiro-Gomez, Quijano, & Ocampo-Martinez, 2014a). The method consists in considering the global problem as a society, where there is limitation of information sharing. The society is divided into several populations, where there is full available information. Then, a local optimization problem is solved at each population whose feasible region varies dynamically, i.e., there is an interchange of masses among populations. The feasible regions vary until all populations agree to solve the global optimization problem. In addition to this, applications in the control field may involve disturbances that could lead the trajectories to leave the feasible region (given by constraints that impose a desire performance) (Barreiro-Gomez, Quijano, & Ocampo-Martinez, 2014b). Another relevant difference with respect to already published distributed population dynamics approaches is that the proposed method guarantees that the feasible region is attractive. The last mentioned feature potentially improve the control performance rejecting disturbances. Finally, the design of the decision-making distributed system allows to have a reduced number of modifications when the graph topology changes, i.e., there are new nodes/edges in the graph or there are nodes/edges that disappear. Also, some redundant links can be identified, i.e., links in the graph that are not necessary in the connection among cliques.

The remainder of the paper is organized as follows. Section  2 shows preliminaries of graphs, population dynamics, and introduces the mathematical formalism that is used throughout the paper. Section  3 presents the population dynamics and the mass dynamics, including relevant characteristics. Then, the stability analysis of the dynamics is presented in Section  4. Section  5 shows the different possible changes that the social graph might suffer, and explains the implication over the design. Section  6 presents the optimization problem forms that could be solved with the population dynamics and the mass dynamics, presenting also some illustrative examples and results. Afterwards, the robustness of the method is shown by applying disturbances in Section  7. Then, Section  8 presents a large-scale system and the design of optimal controllers by using the proposed methodology. Controllers consider both a model-based approach, and a model-free approach. Section  9 shows the results and discussion about the performance of controllers designed with the proposed methodology. In Section  10 the main conclusions are drawn.

Notation

The sub-index is associated to a node of a graph, or to a strategy in a game. On the other hand, the super-index refers to a population. For instance, the sub-index i in xi, Pi, xip or Fi refers either to a node in a graph or to a strategy, and the super-index p in mp, xp, xip or Np indicates a population. Also it should be clear that the super-index is not an operational number, i.e., N3 refers to population three but N3NNN. We use bold font for column vectors and matrices, e.g., x, and H; and non-bold style is used for scalar numbers, e.g., Np. Calligraphy style is used for sets, e.g., S. The column vector with N unitary entries is denoted by 1N, and the column vector with null entries and suitable dimension is denoted by 0. The identity matrix with dimension N×N is denoted by IN. The cardinality of a set S is denoted by |S|. The continuous time is denoted by t, and it is mostly omitted throughout the manuscript in order to simplify the notation. Finally, R+ represents the set of all non-negative real numbers, and Z+ represents the set of positive integer numbers.

Section snippets

Preliminaries

Let G=(V,E) be an undirected non-complete connected graph that exhibits the topology of a society, where V is the set of vertices of G that represents the set of N available strategies in a social game denoted by S={1,,N}; and E{(i,j):i,jV} is the set of edges of G that determines the possible interactions among society strategies. The graph G is divided into M sub-complete graphs known as cliques (a complete sub-graph), where each clique represents a population within the society. The set

Population and mass dynamics

The objective for the society is to converge to a Nash equilibrium1 of the game F denoted by xΔ. In order to achieve this objective, there is a game at each population pP converging to a Nash equilibrium of the game Fp denoted by xpΔp, and the intersection nodes iI

Stability analysis

It is necessary to show that the solution of the distributed system with population dynamics (6a), and mass dynamics (7a) at intersection nodes, implies the solution of the social game (i.e., the global problem).

Proposition 2

If   Assumption  3   is satisfied, the population dynamics   (6a)   are in equilibrium (xpΔp,for allpP), and the mass dynamics   (7a)   are in equilibrium (mi,for all   iI); then, the society is in equilibrium, i.e., xΔ.

Proof

The equilibrium xpΔp of the population dynamics (6a), for

Changes in the graph

Network systems in engineering are constantly growing. On the other hand, some systems isolate a segment of the network under some specific conditions (e.g., an operation fault), and this might suppose a reduction in the graph. When any of these two situations occur, the controller should be re-designed in order to fit the new system conditions. The method proposed in this paper allows to reduce the number of changes in the original design when changes in the graph are made. It is necessary to

Optimization problems with constraints

One of the main features of full potential games is that their Nash equilibrium points coincide with the extreme points of the corresponding potential function, i.e., Nash equilibria satisfy the KKT first-order conditions (Sandholm, 2010). Additionally, if the potential function is concave, potential games are stable and an optimization problem can be solved in a distributed way by using the population dynamics and the mass dynamics shown in Section  3. Some optimization problem forms are set

Performance against disturbances and the convergence factor

As it has been shown in Section  3, the convergence factor β forces trajectories towards the feasible region. This fact makes the proposed methodology robust against disturbances. In order to illustrate this feature, Example 6.1.1 is solved under different conditions. First, the initial condition does not belong to the feasible region (i.e., x(0)Δ). Additionally, as a second factor affecting the evolution of states, a hard disturbance is applied to a node in the graph shown in Fig. 5. The

Case study

In an optimal control design, the control actions are commonly the decision variables. Then, from the point of view of our proposed approach, the proportion of agents playing each strategy is associated to each control action of the system. It is guaranteed that the solution obtained with the proposed approach satisfies the established constraints. However, due to the fact that trajectories of proportion of agents might get out from the feasible region in a transitory event, it is not

Results and discussion

In order to analyze the proposed methodology, four different scenarios are proposed, two of them designed with the proposed population and mass dynamics approach and two of them by using a centralized MPC approach:

  • Scenario 1: controller based on the proposed population and mass dynamics, and considering a COM.

  • Scenario 2: model-free controller based on the proposed population and mass dynamics.

  • Scenario 3: MPC controller with perfect COM (i.e., the COM equal to the SOM) and with a prediction

Concluding remarks

A methodology to solve different optimization problems with multiple constraints has been presented. The method is based on population dynamics, whose set of states varies over time (i.e., dynamics over the population masses are added). The variation of the set of possible states represents a mass interchange among populations. It has been shown that the population dynamics and the mass dynamics are stable and that the feasible region of the global problem is attractive, under the presented

Acknowledgment

We would like to thank Germán Obando for his inputs and the academic discussions about this work.

Julian Barreiro-Gomez received his B.S. degree in Electronics Engineering from Universidad Santo Tomas (USTA), Bogotá, Colombia, in 2011. He received the M.Sc. degree in Electrical Engineering from Universidad de Los Andes (UAndes), Bogotá, Colombia, in 2013. Since 2012, he is with the research group in control and automation systems (GIAP, UAndes), where he is pursuing the Ph.D. degree in Engineering in the area of control systems. Since 2014, he is an associate researcher at Technical

References (31)

  • Grosso, J.M., Ocampo-Martinez, C., & Puig, V. (2012). A service reliability model predictive control with dynamic...
  • J. Hofbauer et al.

    Stable games and their dynamics

    Journal of Economic Theory

    (2009)
  • Jadbabaie, A., Ozdaglar, A.E., & Zargham, M. (2009). A distributed Newton method for network optimization. In...
  • Johansson, B., Keviczky, T., Johansson, M., & Johansson, K.H. (2008). Subgradient methods and consensus algorithms for...
  • H.C. Johnston

    Cliques of a graph-variations on the Bron-Kerbosch algorithm

    International Journal of Parallel Programming

    (1976)
  • Cited by (44)

    • Convergence analysis of distributed population dynamics based on second-order Delaunay triangulation

      2022, Information Sciences
      Citation Excerpt :

      Among various distributed population dynamics, distributed Smith dynamics have been intensively investigated for the capability to properly handle the non-negativity constraints due to its physical nature in various practical problems [27]. Most studies pay attention to modeling the distributed population dynamics for a given communication network or satisfying specific communication networks [47,1,16]. Yet the relationship between communication networks and the convergence of distributed population dynamics is not entirely understood.

    • Nash equilibrium seeking in full-potential population games under capacity and migration constraints

      2022, Automatica
      Citation Excerpt :

      Regarding GNE seeking problems in evolutionary games, on the other hand, different approaches have been recently proposed. Namely, Barreiro-Gomez, Quijano, and Ocampo-Martinez (2016) consider the migration-constrained population games of Barreiro-Gomez et al. (2017) in the context of density-dependent population games (i.e., population games where the total mass of players is not constant), and propose a primal–dual approach to include affine constraints over the strategic distribution of the population. The proposed approach is illustrated on distributed GNE seeking problems in water distribution systems.

    • A payoff dynamics model for generalized Nash equilibrium seeking in population games

      2022, Automatica
      Citation Excerpt :

      Some recent approaches that shed light onto this topic are the ones in Barreiro-Gomez et al. (2016), Barreiro-Gomez and Tembine (2018), and Martinez-Piazuelo et al. (2022). Namely, Barreiro-Gomez et al. (2016) introduce the concept of mass dynamics to consider affine constraints in stable population games under certain imitative revision protocol; Barreiro-Gomez and Tembine (2018) propose some novel revision protocols to consider affine box inequality constraints in stable games; and Martinez-Piazuelo et al. (2022) propose some dynamic payoff mechanism for GNE seeking in potential population games with affine equality constraints and under impartial pairwise comparison revision protocols. In contrast with the aforementioned previous works, in this paper we consider the problem of GNE seeking in population games under fairly general constraints.

    • A payoff-based learning approach for Nash equilibrium seeking in continuous potential games

      2022, Neurocomputing
      Citation Excerpt :

      In this case, the only available information of each agent is its own past actions and corresponding utilities. The lack of full information may be encountered in a variety of scenarios [20–24]. For instance, in motion coordination problems [25–27], agents may have an ability to sense its distance with other agents, but they may not have an ability to sense other agents’ relative or absolute positions.

    View all citing articles on Scopus

    Julian Barreiro-Gomez received his B.S. degree in Electronics Engineering from Universidad Santo Tomas (USTA), Bogotá, Colombia, in 2011. He received the M.Sc. degree in Electrical Engineering from Universidad de Los Andes (UAndes), Bogotá, Colombia, in 2013. Since 2012, he is with the research group in control and automation systems (GIAP, UAndes), where he is pursuing the Ph.D. degree in Engineering in the area of control systems. Since 2014, he is an associate researcher at Technical University of Catalonia (Barcelona, Spain), Automatic Control Department (ESAII), and Institut de Robòtica i Informàtica Industrial (CSIC–UPC), where he is pursuing the Ph.D. degree in Automatic, Robotics and Computer Vision. His main research interests are constrained model predictive control, distributed optimization and control, game theory, and population dynamics.

    Nicanor Quijano received his B.S. degree in Electronics Engineering from Pontificia Universidad Javeriana (PUJ), Bogotá, Colombia, in 1999. He received the M.S. and Ph.D. degrees in Electrical and Computer Engineering from The Ohio State University, in 2002 and 2006, respectively. In 2007 he joined the Electrical and Electronics Engineering Department, Universidad de los Andes (UAndes), Bogotá, Colombia as Assistant Professor. In 2008 he obtained the Distinguished Lecturer Award from the School of Engineering, UAndes. He is currently Full Professor and the director of the research group in control and automation systems (GIAP, UAndes). On the other hand, he has been a member of the Board of Governors of the IEEE Control Systems Society (CSS) for the 2014 period, and he was the Chair of the IEEE CSS, Colombia for the 2011–2013 period. Currently his research interests include: hierarchical and distributed optimization methods using bio-inspired and game-theoretical techniques for dynamic resource allocation problems, especially those in energy, water, and transportation. For more information and a complete list of publications see: https://profesores.uniandes.edu.co/nquijano/.

    Carlos Ocampo-Martinez received his electronics engineering degree and his M.Sc. degree in industrial automation from the National University of Colombia, Campus Manizales, in 2001 and 2003, respectively. In 2007, he received his Ph.D. degree in Control Engineering from the Technical University of Catalonia (Barcelona, Spain). In 2007–2008, he held a postdoctoral position at the ARC Centre of Complex Dynamic Systems and Control (University of Newcastle, Australia) and, afterwards at the Spanish National Research Council (CSIC), Institut de Robòtica i Informàtica Industrial, CSIC–UPC (Barcelona) as a Juan de la Cierva research fellow between 2008 and 2011. Since 2011, he is with the Technical University of Catalonia, Automatic Control Department (ESAII) as Associate Professor in automatic control and model predictive control. Since 2014, he is also Deputy Director of the Institut de Robòtica i Informàtica Industrial (CSIC–UPC), a Joint Research Center of UPC and CSIC. His main research interests include constrained model predictive control, large-scale systems management (partitioning and non-centralized control), and industrial applications (mainly related to the key scopes of water and energy).

    This work has been partially supported by the projects “Drenaje urbano y cambio climático: hacia los sistemas de alcantarillado del futuro. Colciencias 548/2012”, and ECOCIS (Ref. DPI2013-48243-C2-1-R). Julian Barreiro-Gomez is supported by COLCIENCIAS-COLFUTURO (grant 6172) and by the Agència de Gestió d’Ajust Universitaris i de Recerca AGAUR (grant FI-2014). The material in this paper was partially presented at the 53rd Conference on Decision and Control, December 15–17, 2014, Los Angeles, CA, USA. This paper was recommended for publication in revised form by Editor Berç Rüstem.

    View full text