Game-theoretic beamforming techniques for multiuser multi-cell networks under mixed quality of service constraints

We propose a game-theoretic approach for the downlink beamformer design for a multiuser
multi-cell wireless network under a mixed quality of services criterion. The network has real time users that must attain a specific set of signal-to-interference-plus-noise ratios, and non-real time users whose signal-to-interference-plus-noise ratios should be balanced and maximized. We
propose a mixed QoS strategic noncooperative game wherein base stations determine their downlink beamformers in a fully distributed manner. In the case of infeasibility, we have proposed a fall back mechanism which converts the problem to a pure max-min optimization. We further propose the mixed QoS bargain game to improve the Nash equilibrium operating point through Egalitarian and Kalai-Smorodinsky bargaining solutions. We have shown that the results of bargaining games
are comparable to that of the optimal solutions.


Introduction
In recent years, game-theoretic algorithms have been widely studied for understanding interaction of wireless devices and to design distributed resource allocation methods. Game theory is a discipline in economics, which is used to model and analyse situations where decision makers may have conflicting interests [1,2]. The conflicting nature of transmitters in a multiuser wireless network makes it relevant to represent the optimisation problem from a game-theoretic perspective. Game-theoretic solutions offer a structure that readily allows for decentralised implementation.
Game theory was applied to a power control problem for a single-input single-output model in [3][4][5]. The utility function was modelled as the number of information bits received per Joule of energy expended which captures the trade-off between quality of service (QoS) requirements and energy consumption. The latter work was extended to a multi-carrier data network in [6]. Noncooperative games were used in [7][8][9][10] to develop distributed algorithms. Demonstrating the inefficiency of the Nash equilibrium with respect to Pareto optimality, a pricing mechanism was introduced in [7,8] to obtain efficient solutions; hence, this required cooperation among the players.
A multiple-input multiple-output interference channel setup was considered in [9][10][11]. Both the non-cooperative and the cooperative game-theoretic methods were considered in [10]. The cooperative games proposed in [12][13][14][15][16] were able to achieve Pareto optimal solutions. In [15], bargain theory was used to derive Kalai-Smorodinsky solution to maximise sum data rate while ensuring users get a share of the achieved sum rate in the same proportion that they would achieve in the absence of any interference. Suris et al. in [15] compared Nash bargain (NB) solution to both the Kalai-Smorodinsky and the Egalitarian solutions and concluded that the NB solution achieves a better trade-off between fairness and efficiency. Competitive and coordinated beamformer design methods for a multicell downlink network were proposed in [17]. Another related work in [18] considered signal-to-interferenceplus-noise ratio (SINR) balancing for multiuser multi-cell network using a game-theoretic approach.

Contributions
Further to the works discussed above, we propose beamformer design based on mixed QoS criterion using both the noncooperative (competitive) and the cooperative (bargaining) games. The specific contributions of this work are as follows: • Considering a situation where there are two groups of users, namely real time users (RTUs) and non-RTUSs (NRTUs), we have proposed and solved beamformer design techniques using game theory to meet two different classes of quality of services.
As the RTUs are very sensitive to delay, the SINRs for these users are satisfied with a set of specific target SINRs. The SINRs for the NRTUs are allocated fairly based on a max-min criterion (SINR balancing). • The existence of the Nash equilibrium has been shown for the proposed mixed QoS strategic non-cooperative game (SNG). • To address the issue of infeasibility, a fallback mechanism has been proposed based on allocation of data rates in the same proportion as that users would obtain if there is no inter-cell interference. • We propose the Egalitarian and Kalai-Smorodinsky bargaining solutions to improve the Nash equilibrium operating point.

System model and problem formulation
Consider a multi-input single-output downlink consisting of a set = [1, …, N] base stations (BSs) and a set = [1, …, K] singleantenna mobile stations (MSs). Each MS is attached to at most one BS at any given time as shown in Fig. 1. Let us denote the index of the BS that serves the kth user by n k , the set of all users served by the nth BS as n , the set of all RTUs served by the nth BS as n R ⊂ n and a set of all NRTUs served by the nth BS as n N ⊂ n . The cardinalities of the sets n , n R , and n N are K n , K1 n , and K n − K1 n , respectively. It is assumed that all BSs share the same frequency band. Each BS is equipped with M antennas. The maximum transmission power at nth BS is limited to p n max . The transmitted signal for the kth user from the BS n k is written as where s k (t) ∈ ℂ represents the information symbol at time t and w k ∈ ℂ M × 1 is the transmit beamforming vector for the kth user. We assume that s k (t) is normalised such that { | s k (t) | 2 } = 1 as the power of the signal can be incorporated into w k . All data streams are assumed to be independent, i.e. {s k (t)s i (t) * } = 0 if k ≠ i. The received signal at the kth user can be written as where h n k , k ∈ ℂ M × 1 is the channel vector from the BS n k to user k, and z k (t) ∈ (0, σ 2 ) is the circular symmetric zero mean complex Gaussian noise with variance σ 2 . The notation ∖{k} denotes a set excluding the member k.

Problem formulation
As in (2), all the users experience inter-cell interference. The instantaneous downlink SINR of the kth MS is We denote the precoding matrix at the nth BS as W n = [w 1 , …, w K n ]. The feasible set of beamformers of the nth BS is given by where ξ k is the SINR target of the kth RTU. The QoS feasible region ⊂ ℝ + K is The QoS feasible region at the nth BS, n ⊂ ℝ + K n is a subset of given by We want to form a strategic non-cooperative game where each BS is only aware of its QoS feasible region n . As the RTUs have a specific set of SINR to attain, we consider the utility of the game as the worst-case SINR (balanced SINR) of the NRTUs. The mixed QoS sum rate is qualified when the SINR of the worst-case NRTUs is maximised while guaranteeing SINR targets for the RTUs. We will be applying in iterative methods; hence, let us define the intermediary SINR target of the kth NRTUs as δ k . The global optimisation problem OPTG(n) is formulated as Problem in (7) shows that the feasibility depends solely on the feasibility of satisfying the SINR targets of the RTUs under the power constraint. The feasibility of (7) can be evaluated by a power minimisation problem as discussed later.

Strategic non-cooperative game
Each BS is the player that is aware of only its local channel state information (CSI), i.e. the channels of its serving users. This CSI is assumed to be private. The set of beamformers W n is the strategy of each player n (BS) and the SINR of the worst-case NRTUs is the utility. We denote the inter-cell interference plus the noise power at the kth MS as where W −n are the beamformer vectors of all other BSs except that of the nth BS. At each BS, we will have r −n = [r −n 1 , …, r −n K n ] T . The main motivation of representing the inter-cell interference plus noise by r −n is to enable BSs to perform distributed optimisation since it decouples the strategy sets. Thus, the problem in (7) will be separable in n ∈ . Initially each BS approximate r −n k to σ 2 and implement the required beamformers. Subsequently, each user will feedback r −n k to the serving BSs. The intention of each BS is to maximise the SINRs of the NRTUs while satisfying the SINR targets for the RTUs. It has been proven in [19] that all NRTUs will attain identical SINR; hence, all the intermediary SINR targets of the NRTUs are denoted with an identical value Δ n NRTU . The overall mixed QoS SNG is described as The QoS constraints in (7) can be rewritten in their equivalent second-order cone (SOC) form as where ℜ( ⋅ ) extracts the real part of the argument. Since (7) is quasi-convex [20], it is solved using a bisection search [21]. Each play round of the proposed game consists of two stages, namely the qualification stage and the learning stage (see Fig. 2a). In the qualification stage, each player separately performs a bisection search on the feasible region n to determine the optimal beamformer W n that solves (7) distributively for a given r −n as follows; each player performs a bisection search on Δ n NRTU by solving OPT n : minimise Our aim is to obtain solutions of OPT n , ∀n ∈ , which will solve OPTG(n). The optimal solution of (7) is obtained by solving (11) for a given Δ n NRTU , however iteratively obtaining the maximum possible Δ n NRTU . It should be noted that for a fixed r −n , this stage requires only the knowledge of the feasible region n . The learning stage follows implementation of these beamformers and computation of the new r −n .

Existence of Nash equilibrium of the sub-game
Since we have used the background noise vector r −n to decouple the strategy set of the game, we invoke the Debreu-Fan-Glicksberg theorem [22,23] to demonstrate the existence of the Nash equilibrium for the game in (9). According to [22,23], the strategy spaces S n ∀n ∈ of a SNG are non-empty, compact and convex sets and the utility function u n (s) is a continuous function in the profile of strategy s and quasi-concave in strategy of nth player s n , then the game has at least one pure Nash equilibrium [22,23]. The problem we consider admits at least one Nash equilibrium for the following reasons. The strategy profile of n is a convex set as shown in (4). Since constraint on SINR k is a SOC and is quasi-convex on the set n , the Debreu-Fan-Glicksberg is satisfied. Therefore, game is quasi-concave and it has at least one pure Nash equilibrium.

Determining the pure Nash equilibrium of the sub-game
For brevity, let us denote the SINR targets of all RTUs and the intermediary SINR targets of all the NRTUs at the nth BS as Γ n = [γ 1 , …, γ K n ]. By assuming that the maximum possible worstcase NRTU SINR Δ n NRTU ⋆ is known from the previous round of the game, the optimal beamformer vector w k ⋆ , k ∈ n [17,24] will be obtained as where p k is the beamforming power, w k ⋆ is the unit-norm beamforming vector for the kth user, and λ k ≥ 0 is the Lagrange multiplier associated with the kth SINR constraint in (11). These Lagrange multipliers can be obtained by fixed point iteration as [25] Hence the problem in (11) can be written as The transmission power to the users at the nth BS are stacked in a power allocation vector p n = [p 1 , …, p K n ] T ∈ ℝ . We define the SINR in terms of p n and the interference function where The constant link gain matrix (i.e. a coupling matrix) Ψ n for the nth BS is defined as At every bisection level, let us put the SINR targets of all users served by the nth BS in a diagonal matrix as D n = diag{γ 1 , …, γ K n }.
According to the analysis in [25,26], there exists an optimiser for (13) given by a fixed-point iteration where i is the time (iteration) index. This provides an optimiser for each BS during qualification stage. During every play round, players will adaptively change their beamformers, and a new r −n will be measured hence the best response (BR) of each player is to solve (17) repeatedly for every new values of r −n . As each BS knows its maximum possible SINR Δ n NRTU ⋆ of the NRTUs for a given r −n from the previous iteration, then the BR of the nth BS at each game round is defined as Hence (7) will have an optimiser at the maximum possible SINR target D n ⋆ . The game will converge when the newly learned r −n causes no change to the previous power vector p n . If at any stage, the qualification becomes infeasible for a given r −n , and in order to force the game to converge, it is assumed that the affected players will be forced to adopt their previous feasible beamformers and power allocation.

Fallback mechanism
Unlike the conventional max-min problem, the mixed QoS SNG may become infeasible if the SINR targets of the RTUs will not be satisfied. This could be due to inadequate transmission power and/or bad channels. Consequently, for the case of multiple BSs, if the mixed QoS problem turns out to be infeasible for any of the BSs, then the definition of the game in (9) will be amended using a fallback mechanism. Accordingly, if the mixed QoS power allocation in (11) becomes infeasible during the first play round/ qualification stage, then only max-min problem will be used for both the RTUs and NRTUs of that corresponding BS. Alternatively, BS may opt to drop certain RTUs by solving the admission control problem [27]. As this totally excludes the dropped users from taking part in the bargaining process, this approach is not considered in this paper. The steps for the mixed QoS SNG are summarised in Algorithm 1 (see Fig. 3).
The mixed QoS SNG described in Algorithm 1 (Fig. 3) includes the fallback mechanism option in step 15. In Algorithm 1 (Fig. 3), the qualification stage (the first play round), involves each BS solving (11) with r −n k = σ 2 , ∀k (i.e. ignoring interference from other BSs) iteratively using a bisection method. This stage will determine if there is a need for the fallback mechanism.

Mixed QoS bargaining games
We consider a cooperative (bargaining) game under mixed QoS criterion in (9). Since the global optimisation problem in (7) is concerned with fairness, we develop a mixed QoS Egalitarian bargaining game and mixed QoS Kalai-Smorodinsky bargain game. In the mixed QoS Egalitarian bargain game, the players will have equal share of the remaining feasible region enclosed between the Nash equilibrium and the Pareto boundary, denoted as + . The mixed QoS Kalai-Smorodinsky bargain game on the other hand allows players to acquire a share that is proportional to what they would achieve in the absence of interference. We adopt the alternating direction method of multipliers (ADMM) [28,29] to develop distributed algorithms for these bargaining solutions.

Mixed QoS Egalitarian bargain game
The mixed QoS Nash equilibrium point is in general inefficient [3][4][5]. By sharing certain information between the players, Pareto optimal results can be achieved. In [28], a decentralised algorithm under SINR balancing criterion was developed using ADMM. We adopt this approach to develop an algorithm using the mixed QoS criterion. Unlike in [28] where the searching space of the algorithm is defined from the origin to the Pareto optimal boundary, we demonstrate that during bargaining, the search space can be reduced starting from the mixed QoS Nash equilibrium point instead of the origin. We denote the achieved SINRs at Nash equilibrium at the nth BS as Γ n NE = [γ 1 NE , …, γ K n NE ]. The global system Egalitarian problem is formulated as where α k is either a 0 or 1. When α k is 0, then the term SINR k /α k will go to infinity. This will automatically exclude the corresponding SINR from the objective of the optimisation problem. Ideally, if the SINR targets of the all the RTUs at each BS are attained at the Nash equilibrium, then the values of α k will be equal to 0 for all RTUs and to one for all NRTUs. In this case, NRTUs will have equal share on + . Even if the fallback mechanism is in place, certain RTUs may not be able to achieve their target SINRs due to bad channel conditions. For this case, the players would give the affected RTU a high priority during bargaining by initially setting α k = 1 for the affected RTU and α k = 0 for all NRTUs. Once the RTUs reach their SINR targets, their SINRs will be fixed and the remaining resources will be shared equally amongst the NRTUs by setting their α k to 1. We then perform a mixed QoS optimisation in + . Our proposed mixed QoS Egalitarian bargain game maximises the minimum SINR of the NRTUs at each BS while satisfying SINR targets for the RTUs. We denote the optimal SINRs from Egalitarian bargain game solution at each BS as Γ n EBG = [γ k EBG , …, γ K n EBG ]. The optimal mixed QoS Egalitarian bargain game solution will allocate equal share of resource to the NRTUs given as This means that we expect the optimal SINRs of the NRTUs at each BS to give equal value c n if the RTUs achieved their SINR targets at the Nash equilibrium. If this is not the case, the values of c n may be different because the affected RTUs will have a share on + .

Mixed QoS Kalai-Smorodinsky bargain game
In The values c n and v n in (20) and (22) reveal the relationship between the Egalitarian bargain game and Kalai-Smorodinsky bargain solutions. Therefore we will derive an algorithm that will provide solution to both the mixed QoS Egalitarian and Kalai-Smorodinsky bargain games.

System model for bargaining solutions
Consider the system in Section 2. To keep the distributed nature of the game-theoretic solution, we aim to reduce the information shared between the BSs using a global consensus approach and derive the augmented Lagrangian problem. We introduce auxiliary variables κ n, k , and κ n, k [28,29] to represent the actual inter-cell interference from the nth BS to user k and its local copy at the nth BS, respectively. This allows a consistency constraint between the global and local copies to ensure that κ n, k and κ n, k are in consensus.
The overall feasibility test of the network is performed by the following global consensus problem: κ n, k ≥ κ n, k , k ∈ n , n ∈ , n ≠ n k .
The above problem can be transformed into a convex SOC programming (SOCP) by concatenating all the beamforming vectors at the nth BS into a matrix W n = [w k ] k ∈ n as follows: k ∈ n , n ∈ , n ≠ n k , κ n, k ≥ κ n, k , k ∈ n , n ∈ , n ≠ n k , where κ k = {κ n, k } n ∈ ∖n k and the notation ⪰ SOC refers to the generalised inequalities with respect to the SOC [21]. The problem in (24) is convex and separable in n ∈ allowing a distributed power minimisation based on the ADMM algorithm as in [28,29]. With reference to the third constraint in (24), we define column vectors κ n by concatenating all local variables associated with the nth BS and κ n by concatenating all global variables associated with elements of κ n . Thus, for fixed global variables κ n , the set of beamformers at the nth BS given in (4) depends on the local variables κ n . Taking this into account, we form the feasibility indicator function f n W n , κ n at the nth BS as which leads to a compact model of problem in (24) defined by minimise ∑ n ∈ f n W n , κ n subject to κ n ≥ κ n , ∀n ∈ .
where s is the iteration index. More details for the solution of (28)-(30) as presented in Algorithm 2 (see Fig. 4) can be found in [28,29]. For a fixed global variable κ, iteration in (28) is solved at each BS by solving the following problem: In (29), BSs gather κ s + 1 from their neighbours to form averages. In essence, since each element of the global variable couples two local variables of the neighbouring BSs, its solution is simply the average of its local variables given by κ n, k s + 1 = κ n k , k s + 1 + κ n, k s + 1 /2.

Convergence of the bargaining game
As in [29], the critical assumptions required for the cooperative game to converge is that f n W n , κ n is closed, proper, convex and the unaugmented Langrangian ℒ 0 has a saddle point. Under these assumptions, the iterations of ADMM satisfy the following: i. The dual residual (κ n s − κ n s ) → 0 as s → ∞.
ii. The objective function in (31) converges, i.e. f n W n , κ n → W ⋆ as s → ∞. iii. The dual variable converges, i.e. y n s → y n ⋆ where y n ⋆ is the dual optimal point. iv. The local variable κ n s and the global variables κ n s do not necessarily have to converge to the optimal values [29].
The indicator function in (25) is closed, proper, and convex; therefore, it satisfies the characteristics defined above. This means that the ADMM algorithm will converge to the globally optimal solution, and if there is a unique point that will be reached as s approaches ∞.

Numerical examples
To illustrate the performance of Algorithms 1 and 2 (Figs. 3 and 4), we use a network topology as shown in Fig. 1 (also refer to Fig. 2). The network has two BSs (i.e. two players) each equipped with M = 4 antennas. The maximum transmission power at each BS is p n max = 20 dBW. Each BS serves three single antenna MSs that include two NRTUs and one RTU. The SINR targets for the RTUs at the first BS and the second BS were set to 10 and 12 dB, respectively. The line search accuracy at each BS is set to δ = 10 −4 . The maximum iteration is set to 15 for the mixed QoS strategic non-cooperative game and all bargaining games. All channel vectors, h n k , k , are chosen from the distribution (0, I) which resembles uncorrelated frequency flat Rayleigh fading. The noise power was set to σ 2 = 0 dBW for all users. We used the centralised solutions as a reference system for performance comparison. Under the mixed QoS framework, there are two main possible scenarios: i. The SINR targets of both the RTUs are satisfied.
ii. The SINR target of at least one RTU is not satisfied because of: (a) Inter-cell interference.
(b) Or during the first play round, at least one of the BSs fails to pass the qualification stage. Using the fallback mechanism, the affected BS then considers a pure max-min problem. This will introduce interference to the other BSs and may encourage other BSs to consider bargaining.

Results under scenario 1
We study the outcome of scenario 1 in Figs. 5a and b. We note that at the mixed QoS Nash equilibrium, both RTUs achieved their SINR targets and the SINRs of NRTUs were balanced. The convergence rate of the mixed QoS SNG is slow under this scenario. This is because, since both BSs are able to achieve the SINR target of their RTUs at each qualification stage, every time a new value r −n is observed, the BSs will have to adapt their beamformers. The SINRs achieved by the mixed QoS SNG and mixed QoS Egalitarian bargain game were compared to the centralised Egalitarian solution. We observe that under the mixed QoS SNG, all the NRTUs achieved the balanced SINRs that are lower than the balanced SINRs they will achieve under the centralised Egalitarian solution. However, during the mixed QoS Egalitarian bargain game, the NRTUs at BS1 achieved balanced SINRs that are more than the balanced SINRs provided by the centralised Egalitarian solution. In the mixed QoS Egalitarian bargain game, all the NRTUs improved their Nash equilibrium SINRs by an equal amount. Similar trends are observed in Fig. 5b. It is noted that in the centralised Kalai-Smorodinsky solution, the SINRs of NRTUs are not necessarily balanced to the same SINR value.

Results under scenario 2
In Fig. 6a, we note that during the mixed QoS SNG, only BS2 managed to balance the SINRs of its NRTUs while achieving the SINR targets of its RTU. The BS1 failed to achieve the SINR target of its RTU due to inter-cell interference. Therefore, BS1 adapts its previous beamformers and power allocation which forces the game to converge in the subsequent iteration. During the mixed QoS Egalitarian bargain game, the BS1 managed to achieve the SINR target of its RTU during the first iteration of bargaining. The remaining resources were shared fairly amongst all the NRTUs. Similar trends are observed in Fig. 5b, but it takes longer time for the RTU at BS1 to converge to its SINR target.

General performance of the proposed algorithms
A further analysis on the performance of mixed QoS SNG Algorithm 1 (Fig. 3) and mixed QoS bargaining Algorithm 2 ( Fig. 4) was performed using 500 random channel realisations. In Fig. 7, we depict the achieved SINRs by all users under various solutions at BS1 and BS2. In Fig. 7a, we observe that both the centralised Egalitarian and Kalai-Smorodinsky solutions are able to achieve the SINR target of the RTUs for all channel realisations. Further, the centralised solutions always balance the SINRs of the NRTUs. We note that the mixed QoS SNG failed to achieve the SINR target of the RTUs for one of the channel realisations. The mixed QoS SNG is inefficient as compared to the centralised solutions. The mixed QoS Egalitarian bargain game managed to achieve the SINR target for all channel realisations at BS1. Similar trend is observed in Fig. 7b. We notice that the BS2 attained higher SINRs that are below 5 dB as compared to BS1. This is because, the SINR target of the RTU at BS2 is larger than the SINR target at BS1. This draws more transmission power for the RTU at BS2 as compared to the NRTUs. Nevertheless, both BSs are able to achieve higher SINRs that are greater than 20 dB as compared to the centralised solutions.
The mean and the variance of the sum rate for the mixed QoS problem are summarised in Table 1. As anticipated, in terms of mean, the mixed QoS SNG is inefficient compared to all the centralised and bargaining solutions. Both the bargaining solutions outperform the centralised solutions at BS1, but this is not the case at BS2. The latter observation comes as a consequence of the high SINR target of RTU2. We note that the variance of the bargaining solutions is larger than those achieved by the mixed QoS SNG and the centralised solutions.  We proposed a game-theoretic framework for the downlink beamformer design in a multi-cell network under mixed QoS criterion. Both the mixed QoS SNG and the mixed QoS bargain game were studied. In the mixed QoS SNG, each BS serves RTUs, who require a specific SINR target and NRTUs whose SINR should be balanced and maximised. We proposed a mixed QoS SNG whereby each BS determines optimal downlink beamformers in a distributed manner by considering estimate of inter-cell interference plus noise power. The mixed QoS strategic noncooperative game can reach Nash equilibrium which is generally inefficient. Hence, we proposed two bargaining solutions, namely, the mixed QoS Egalitarian bargain game and the mixed QoS Kalai-Smorodinsky bargain game that use the Nash equilibrium as a threat point. On average, the mixed QoS sum rate achieved by the bargaining games is comparable to that of the centralised Egalitarian and Kalai-Smorodinsky solutions.