Power Control via Stackelberg Game for Small-Cell Networks

In this paper, power control for two-tier small-cell networks in the uplink is investigated. We formulate the power control problem as a Stackelberg game, where the macrocell user equipment (MUE) acts as the leader and the small-cell user equipments (SUEs) as the followers. To reduce the cross-tier and co-tier interference and also the power consumption of both the MUE and SUEs, we propose to impose a set of costs on their transmit powers and optimize not only the transmit rate but also the transmit power. The corresponding optimization problems are solved by two-layer iterations. In the inner iteration, the SUEs compete with each other and their optimal transmit powers are obtained through iterative computations. In the outer iteration, the MUE's optimal transmit power is obtained in a closed form based on the transmit powers of the SUEs through proper mathematical manipulations. We prove the convergence of the proposed power control scheme, and also theoretically show the existence and uniqueness of the Stackelberg equilibrium (SE) in the formulated Stackelberg game. Simulation results show great improvement of the proposed power control scheme especially for the MUE.

the entire two-tier small-cell network, which is composed of macrocells and a large number of small cells, can be greatly improved [1]. To improve the spectral efficiency, small cells can share the spectrum with macrocells; however, the co-tier and cross-tier interferences will seriously degrade the system performance due to the sharing of the spectrum. In this regard, proper power control in small-cell networks is required to reduce the interference and the power consumption.
Power control is an important research topic that has been widely investigated in the literature [2]- [5]. In two-tier smallcell networks, small cells can be deployed randomly and freely, and game theory has increasingly been used to achieve distributed power control [6], [7]. In [8], the interference dynamics caused by time-varying environment was considered and a robust mean field game was proposed to control the transmit power of SBSs. In [9]- [11], it was shown that a Stackelberg game can provide a suitable framework for modeling the competition in two-tier networks. Specifically, a power control problem was formulated to maximize energy efficiency with minimal information exchange in [9]. In [10], both uniform and non-uniform pricing schemes were proposed to obtain the optimal resource allocation with a tolerable interference power constraint. In [11], a network interference controller was proposed to minimize the sum interference by pricing the power consumptions. However, most of the existing literature only addressed power control in the downlink and ignored power control of both small-cell user equipment (SUEs) and macrocell user equipment (MUE) in the uplink. 1 Moreover, because most of the existing literature addressed power control through a price-based Stackelberg game, they can determine the optimal price and power control for only one type of device. Therefore, if we formulate the power control problem through a Stackelberg game without pricing, the optimal power control for the two types of UEs in the considered two-tier small-cell networks can be determined simultaneously.
Motivated by the aforementioned discussions, we develop a power control scheme taking both SUEs and MUE into account. First, the power control problem of the considered two-tier small-cell networks is mathematically formulated as a Stackelberg game that consists of one leader and multiple followers. Second, the optimization problem is solved through a two-layer iteration. In the inner iteration, the followers compete with each other, and their optimal transmit powers are obtained through iterative computations. In the outer iteration, the optimal transmit power of the leader is calculated based on the transmit powers of the followers. Then, we theoretically show the convergence of the proposed scheme and the existence and uniqueness of the Stackelberg equilibrium (SE) in the Stackelberg game. Finally, our proposed power control scheme is verified through simulations, showing that it greatly improves the performance of the MUE.
The remainder of this paper is organized as follows. In Section II, the system model of the considered two-tier smallcell networks is presented. In Section III, the proposed power control scheme via a Stackelberg game is developed. The simulation results are presented in Section IV. Final conclusions are drawn in Section V.

II. SYSTEM MODEL
Consider the two-tier small-cell network shown in Fig. 1, which consists of one macrocell and K small cells. Assume that the macro base station (MBS) and SBSs share the same spectrum and that only one UE communicates with each BS at any time. In the uplink, each SBS will experience interference from the MUE and its nearby SUEs, and the MBS will experience interference from its nearby SUEs. Let P 0 denote the transmit power of the MUE served by the MBS, P k denote the transmit power of the kth SUE, and p = [P 1 , P 2 , · · · , P k , · · · , P K ] T denote the transmit power vector of the considered K SUEs. Then, the transmit rate of the MUE served by the MBS can be expressed as follows: where H 00 denotes the channel gain from the MUE to its corresponding MBS, H 0k is the interference channel gain from the kth SUE to the MBS, and N 0 is the noise power. The transmit rate of the SUE served by the kth SBS can be expressed as follows: The schematic of the considered two-tier small-cell network.

= ln
where p −k denotes the transmit power vector of the K − 1 other SUEs and p −k = [P 1 , P 2 , · · · , P k−1 , P k+1 , · · · , P K ], H kk is the channel gain from the kth SUE to its corresponding SBS, H k0 is the interference channel gain from the MUE to the kth SBS, and H kk is the interference channel gain from the k th SUE to the kth SBS.
The design objective of this paper is to develop a power control scheme that can increase the transmit rate with reduced co-tier and cross-tier interferences and power consumption. Moreover, this paper aims to achieve the above design objective for two-tier small-cell networks where there are two types of UEs, i.e., MUE and SUE, and two different cell types, i.e., macro cell and small cell.
III. THE PROPOSED POWER CONTROL SCHEME VIA STACKELBERG GAME In this section, we propose a power control scheme for two-tier small-cell networks based on a Stackelberg game, which has one leader and multiple followers. In the formulated Stackelberg game, the MUE, acting as the leader, is supposed to make its own decision and maximize its utility with the best responses of the followers, and the SUEs acting as the followers will respond to the leader's action and maximize their utilities through a subgame [12]- [14]. Note that the transmit power of the MUE or SUE is controlled by its corresponding MBS or SBS and the MBS can control its corresponding SBSs in the considered two-tier small-cell network. When the transmit power of the MUE has been determined by the MBS, this transmit power information will be sent from the MBS to its corresponding SBSs. Therefore, the MUE controlled by the MBS acts as the leader, and the SUEs controlled by its corresponding SBSs act as the followers.

A. Stackelberg Game Formulation
From (1) and (2), we find that the transmit rate of the MUE can be improved by increasing the transmit power of the MUE but at the cost of increased cross-tier interference to the SUEs. Likewise, the transmit rate of the SUEs can be improved by increasing the transmit power of the corresponding SUE but at the cost of increased cross-tier interference to the MUE and increased co-tier interference to the other K − 1 SUEs. To reduce the cross-tier and co-tier interferences and the power consumptions of both the MUE and SUEs, we propose optimizing not only the transmit rate but also the power consumption. First, the leader MUE moves and determines its transmit power. Subsequently, the follower SUEs move and update their power control strategies to maximize their individual utilities based on the MUE's transmit power.
We define the utility function of the MUE as follows: where λ 0 denotes the coefficient characterizing the influence of per unit transmission power for MUE [15]. Then, the optimization problem of the MUE can be expressed as follows: where P T denotes the maximum transmit power of the MUE or SUEs. We define the utility function of the kth SUE as follows: where λ k denotes the coefficient characterizing the influence of per unit transmission power for SUE. Then, the optimization problem of the kth SUE can be expressed as follows: The optimization problems in (4) and (6) lead to a Stackelberg game. In this game, the objective is to find the SE point from which neither the leader nor the followers have incentives to deviate. Just similar to the definition in [10], we define the SE as follows.
Definition 1: Let P * 0 and P * k denote the two solutions for the optimization problems in (4) and (6), respec- is an SE point for the proposed Stackelberg game if the following conditions are satisfied: Generally, the SE for a Stackelberg game can be obtained by finding its subgame perfect Nash equilibrium (NE) [10], [16]. In our proposed Stackelberg game, it can be readily seen that the SUEs compete in a non-cooperative fashion. Therefore, a non-cooperative power control subgame is formulated, where the corresponding NE is defined as the operating point at which no player can improve utility by changing its strategy unilaterally [10].
To obtain the SE of the proposed Stackelberg game, we propose exploiting the backward induction method [17] to solve the above optimization problems. Generally, the followers' best responses can be obtained with the fixed value given by the leader, and then the optimal strategy of the leader can be achieved according to the followers' best responses. Correspondingly, we can first solve the followers' optimization problem in (6). Then, by using the obtained solution, we can solve the leader's optimization problem in (4).

B. The Optimal Solution of the Followers' Optimization Problem
We have the following theorem for the optimal solution of the optimization problem in (6) for the followers.
Theorem 1: Given the transmit power of the MUE, the optimization problem in (6) has a globally optimal solution, as follows: where (·) + ∆ = max(·, 0), and P tmp k denotes the temporary value of the optimal transmit power of the kth SUE.
Proof: As shown, the utility function U k (P k , p −k , P 0 ) is strictly concave. Furthermore, it can be verified that the SUEs' strategy space is a non-empty and close-bounded convex set in Euclidean space. Correspondingly, the optimization problem in (6) can readily be proven to be convex; thus, it has a globally optimal solution. By setting the first-order derivative of U k (P k , p −k , P 0 ) with respect to P k to zero, P tmp k can readily be calculated as shown in (11). By considering the constraint 0 ≤ P k ≤ P T , the optimal solution of the optimization problem in (6) can readily be obtained as shown in (9) or (10). This completes the proof.

C. The Optimal Solution of the Leader's Optimization Problem
After some mathematical manipulations, we can obtain the refined constraint for P 0 according to (9) as follows: where and P min 0,k and P max 0,k are shown at the bottom of next page in (15) and (16), respectively. Definep * = [P * 1 ,P * 2 , · · · ,P * k , · · · ,P * K ] T . Substituting (10) into (3), and after some mathematical manipulations, we obtain where ε k denotes the indicator function with ε k = 1 if P tmp k > 0 and ε k = 0 otherwise, and ε k is the indicator function with ε k = 1 if P T − (P tmp k ) + > 0 and ε k = 0 otherwise. After some further manipulations, the optimization problem for the leader in (4) can be reformulated as follows: where Then, we have the following theorem.
ε k ε k = 0, then the optimization problem in (18) has an optimal solution as shown in (21) at the bottom of this page, where ε k ε k = 0, then B = 0. Take the first-order derivative of U 0 (P 0 ,p * ) with respect to P 0 . Then, we have Set the above expression to zero. Then, we have (28) has no solution. Correspondingly, the objective function of the optimization problem in (18) is definitely a monotonic function, and its solution must be one of the two endpoints. Then, we havẽ If C 2 2 − 4C 1 C 3 ≥ 0, then we can obtain the two solutions of (28), i.e., P 1 0 , P 2 0 . Since the objective function of the optimization problem in (18) is a continuous function, its solution must be among the extreme points and the endpoints. Correspondingly, we havẽ P * 0 = arg max U 0 (P max 0 ,p * ), U 0 (P min 0 ,p * ), U 0 (P 1 0 ,p * ), U 0 (P 2 0 ,p * ) . (30) This completes the proof.

D. The Proposed Power Control Scheme via Stackelberg Game
We are now ready to develop the proposed power control scheme based on the Stackelberg game described in Algorithm 1. 2 In the proposed scheme, the MUE acts as the leader, the SUEs act as the followers, and the Stackelberg game is formed through the two-layer iteration. In the inner iteration, the SUEs compete with each other, and their own transmit powers are updated iteratively based on the transmit power of the MUE, as shown in Theorem 1. In the outer iteration, the MUE updates its own transmit power based on the transmit powers of the SUEs, as shown in Theorem 2. In the proposed power control scheme, each user plays the best response strategy and maximizes its own utility function in each iteration given the chosen transmit powers of the other users in the previous iteration.
Let W denote a K × K matrix whose elements are given by Then, we can establish the following theorem for the convergence of the inner iteration of the proposed scheme. Theorem 3: If the matrix norm of W is not larger than 1, i.e., W ≤ 1, then the inner iteration of the proposed power control scheme via a Stackelberg game as shown in Algorithm 1 converges.
Then, φ(p) can be expressed in a vector-matrix form as follows: Assume that W ≤ 1. Then, we can obtain the following relationship: According to [18], we know that φ(p) is a contraction. Then, according to the Banach contraction theorem introduced in [18], φ(p) has a unique fixed point that is globally asymptotically stable. Correspondingly, the inner iteration of the proposed power control scheme via the Stackelberg game as shown in Algorithm 1 converges. This completes the proof. Note that we can always find a matrix norm of W to satisfy W ≤ 1. Therefore, the convergence of the inner iteration of the proposed power control scheme can always be guaranteed.
In the following, we briefly analyze the convergence of the outer iteration of the proposed scheme. We know that the utility function U k is concave with respect to p k . Therefore, the SUEs can gradually increase their transmit powers from an arbitrary small number to their optima. Then, the obtained transmit power of the SUEs can be used to determine the transmit power of the MUE. When the transmit powers of the SUEs have been increased to their optima, the optimal transmit power of the MUE can then be determined accordingly [19]. For the practical implementation of the proposed Stackelberg game, the SUEs can find their optimal transmit powers by gradually increasing the transmit power until the utility function U k reaches its maximum due to its concave property. Correspondingly, the MUE can always achieve its SE, i.e., the convergence of the outer iteration of the proposed scheme can always be guaranteed.

E. The Existence and Uniqueness of the SE
SE offers a predictable and stable outcome about the transmit power strategies that the MUE and each SUE will choose. For the proposed Stackelberg game, we have the following theorem.
Theorem 4: One and only one SE point exists for the proposed Stackelberg game.
Proof: Generally, we can obtain the SE for the proposed Stackelberg game by finding the NE of its subgame. For the proposed Stackelberg game, there is only one leader. Therefore, the best response of the leader can readily be obtained by solving the optimization problem in (4). At the followers' side, the best response can be achieved by solving the optimization problem in (6). Correspondingly, to prove this theorem, we only need to prove that a unique NE point exists for the subgame at the followers' side.
It can be verified that the SUEs' strategy space is a nonempty and closed-bounded convex set in the Euclidean space. Moreover, it can also be verified that the utility function U k (P k , p −k , P 0 ) is continuous with respect to P k . In addition, the utility function U k (P k , p −k , P 0 ) is concave. According to [20] and [21], we know that the NE exists if the players' strategy space is a non-empty and closed-bounded set in the Euclidean space and the utility function is continuous and concave in its strategy space. Correspondingly, the existence of the NE of the subgame at the followers' side can be proved.
Regarding the uniqueness of the NE, we first state the following lemma [22].
Lemma 1: For a game, if its feasible region is convex and each players' utility function is strictly convex, then the NE of the game is unique.
Then, according to the above-mentioned discussions and the proof of Theorem 1, we can easily verify the uniqueness of the subgame at the followers' side.
This completes the proof.

IV. SIMULATION RESULTS
In this section, the performance of the proposed power control scheme via a Stackelberg game is evaluated via simulations. In the simulations, the radii of the macrocell and small cells are set to be 1000 m and 100 m, respectively. The noise spectral density is set to −174 dBm/Hz. Unless otherwise stated, we set λ = λ 0 = λ k , ∀k. In the following, for description convenience, we useŪ K ,R K , andP K to denote the average utility, the average transmit rate, and the average transmit power of the considered SUEs, respectively, and we useR to denote the average transmit rate of the considered MUE and SUEs.
In Fig. 2 and Fig. 3, we show the utility of the MUE and the average utility of the SUEs of the proposed scheme versus the number of iterations with different K for λ = 10 3 and P T = 0 dBm. As shown, the proposed scheme can converge to a stable state quickly, which verifies that the proposed scheme can converge to the SE. Moreover, both the utility of the MUE and the average utility of the SUEs are observed to decrease with the increased number of the SUEs, which can be attributed to the increased cross-tier or co-tier interference.
In Fig. 4, we show the performance comparison between our proposed scheme and the non-cooperative power control scheme in [23] with λ = 10 3 , P T = 0 dBm, and K = 4. For description convenience, the utility of the MUE and the average utility of the SUEs of the proposed Stackelberg-gamebased power control scheme are referred to as SG-MUE and SG-SUE, respectively. The utility of the MUE and the average utility of the SUEs of the non-cooperative-game-based power control scheme in [23] are referred to as NCG-MUE and NCG-SUE, respectively. As shown, the SG-MUE is close to  and slightly larger than the SG-SUE, and the NCG-MUE is approximately zero and clearly smaller than the NCG-SUE. This result verifies that the proposed scheme can significantly improve the performance of the MUE. 3 In Fig. 5 and Fig. 6, we show the transmit rate of the MUE and the average transmit rate of the SUEs of the proposed scheme versus P T with different K for λ = 10 3 . As shown, the transmit rate first increases with P T when P T is smaller than a certain threshold value, and then it approaches a steady value. When P T is sufficiently small, the MUE and the SUEs are constrained by the maximum transmit power. Correspondingly, their transmit rates are relatively small. As P T increases, their transmit rates increase due to the larger transmit power  constraint. When P T is larger than a certain threshold value, the transmit power will increase, but it will also simultaneously cause more interference. Therefore, the transmit rate of the MUE and that of the SUEs will both stop increasing.
In Fig. 7, we show the transmit rate of the MUE and the average transmit rate of the SUEs versus P T with different λ 0 and λ K for K = 4. As shown, the transmit rate of the MUE (the average transmit rate of the SUEs) is larger when λ 0 (λ K ) is relatively small. The reason for this result is that a smaller cost of the transmit power will stimulate the corresponding player to employ a relatively large transmit power, subsequently resulting in a larger transmit rate.
In Fig. 8, we show the average transmit rate of the MUE and SUEs of the proposed scheme versus λ for K = 4 and P T = 0 dBm. As shown, the average transmit rate remains at a high value when λ is smaller than 30 dB, decreases gradually with the increased λ, and finally remains at a small value. The reason for this behavior is that the MUE or SUE will choose  to decrease the transmit power and also the corresponding transmit rate with the increased λ.
In Fig. 9 and Fig. 10, we show the transmit power of the MUE and the average transmit power of the SUEs versus P T with different λ for K = 4. As shown, the transmit power decreases with λ. Moreover, the transmit power increases with P T when P T is smaller than 0 dBm and remains approximately constant when P T is larger than 0 dBm. The reason for this result is that the maximum transmit power constraint will have no influence on the power control with a sufficiently large P T .

V. CONCLUSIONS
In this paper, we have formulated a power control Stackelberg game for two-tier small-cell networks by considering both the transmit rate and cost. The optimal transmit powers of the MUE and SUEs have been obtained based on the backward induction method. We have developed a two-layer iterative  power control scheme and proven the convergence of this scheme. We have also shown the existence and uniqueness of the SE in the formulated Stackelberg game. Numerical results have been presented to demonstrate the desirable performance of the proposed scheme. For future work, we would like to explore power control with incomplete information for twotier small-cell networks.

DATA AVAILABILITY
The data used to support the findings of this study are included within the article.

CONFLICTS OF INTEREST
The authors declare that they have no competing interests.