Dynamic Resource Pricing and Allocation in Multilayer Satellite Network

: The goal of delivering high-quality service has spurred research of 6G satellite communication networks. The limited resource-allocation problem has been addressed by next-generation satellite communication networks, especially multilayer networks with multiple low-Earth-orbit (LEO) and non-low-Earth-orbit (NLEO) satellites. In this study, the resource-allocation problem of a multilayer satellite network consisting of one NLEO and multiple LEO satellites is solved. The NLEO satellite is the authorized user of spectrum resources and the LEO satellites are unauthorized users. The resource allocation and dynamic pricing problems are combined, and a dynamic game-based resource pricing and allocation model is proposed to maximize the market advantage of LEO satellites and reduce interference between LEO and NLEO satellites. In the proposed model, the resource price is formulated as the dynamic state of the LEO satellites, using the resource allocation strategy as the control variable. Based on the proposed dynamic game model, an open-loop Nash equilibrium is analyzed, and an algorithm is proposed for the resource pricing and allocation problem. Numerical simulations validate the model and algorithm.


Introduction
Research of 6G mobile communication has become a major direction for the upgrade of satellite networks for next-generation mobile communication [1]. This integrated space-air-ground network includes satellite and ground communication. Satellite networks facilitate global communication services and enhance network accessibility in areas inaccessible to ground communication networks [2][3][4][5].
Many companies have launched low-Earth-orbit (LEO) satellites, the next generation in satellite communication [6], for commercial reasons [7]. Telesat launched 28 innovative LEO satellites integrated with a data network. SpaceX and Commsat plan to launch tens of thousands of LEO satellites to build a global satellite Internet. However, the deployment of LEO satellites can affect satellites already in orbit, especially regarding spectrum resource efficiency. Traditional nonlow-Earth-orbit (NLEO) satellites are authorized users with priority to use spectrum resources, and LEO satellites are unauthorized. The use of spectrum resources by LEO satellites interferes with NLEO satellites. Optimum allocation of spectrum resources will help LEO satellites balance user service requirements and address interference problems. The allocation of resources and management of NLEO satellite interference are challenging problems that must be addressed, and research on spectrum resource allocation in satellite communication networks is critically needed [8,9].
Many studies [10][11][12][13] have explored the resource allocation of satellite communication networks. A Stackelberg-game-based resource allocation scheme was proposed to divide satellites into two groups based on their priorities [10]. In another study [11], a beam-hopping scheme was used to maximize the network throughput of a cognitive satellite network. Heuristic algorithms were proposed to solve the spectrum distribution problems. Deep reinforcement learning (DRL) was used to dynamically allocate resources in satellite communication networks [12]. A joint power and sub-channel allocation problem was solved using a novel optimization model [13] to address the interference between primary and secondary networks.
To solve resource allocation problems, a dynamic game-based resource allocation scheme in multilayer satellite communication networks, using resource pricing as the main control variable, is proposed. The contributions of this study follow.
1) A dynamic game-based resource pricing and allocation model for LEO satellites, to control service price and resources for mobile users, is proposed.
2) The resource price is formulated as the dynamic state of the LEO satellites, the variation of which is affected by the resource allocation strategy.
3) Each satellite controls the resource allocation based on the Nash equilibrium solution for the proposed model. The optimal strategy is obtained for each satellite.
The rest of this paper is organized as follows. In Section 2, a system model for the research of a multilayer satellite communication network is provided and the resource pricing and allocation problem formulated. In Section 3, the proposed model is analyzed. Numerical simulations and their results are discussed in Section 4. Section 5 concludes the paper. Fig. 1 shows a multilayer satellite system, multiple mobile users, multiple LEO satellites, and one NLEO satellite. The NLEO satellite is the authorized user, with the right to use the spectrum resources. The LEO satellites are unauthorized users that cannot use the spectrum resources. They only have access to the spectrum resources if the NLEO satellite is not using them. The satellites are running in orbits, and when an LEO satellite moves into the NLEO satellite coverage area, it shuts down communication to allocate spectrum resources to the NLEO satellite. When not in the NLEO satellite coverage area, LEO satellites can use the spectrum resources. In the proposed multilayer satellite system, the movement of the LEO satellites and periodic usage of spectrum resources cause interference in the NLEO satellite network. Thus, controlling the LEO satellite equipment is crucial to managing their service time. This is an effective solution to the interference problem when LEO satellites are in the NLEO satellite coverage area.

System Model
However, the shutdown of LEO satellites will interrupt connected mobile users and increase the waiting time for service. This reduces the quality of service (QoS) and decreases revenue from mobile users. Users whose services are interrupted must be placed in a waiting queue for service recovery, and time in the queue is controlled by LEO satellite service strategies, which are critical to balance QoS and stop interference due to a NLEO satellite. The multilayer satellite system is formulated herein as a dynamic game system using a dynamic variable state and control variables to optimally allocate the LEO satellites. Fig. 1 shows the multilayer satellite system. LEO satellites should control service prices and times to balance the service and interference due to a NLEO satellite.

Problem Formulation
It is assumed that N LEO satellites are in the proposed multilayer satellite system. After paying for spectrum resources, the LEO satellites provide mobile services. The communication load of mobile users on LEO satellite i at time t is denoted m i (t), and the resource allocated by the , u i is the unit cost of service time, and the cost brought by latency is The service price provided by LEO satellite i is denoted p i (t), with i = 1, 2,. . ., N. LEO satellites charge users a constant price c, which is their minimum operation cost, where p i (t) ≥ c. Given the service price, LEO satellites earn a profit of LEO satellites control their pricing strategies, which are considered state variables of the proposed satellite system. The resource allocation strategies are the main criteria used to define resource-pricing strategies because the allocation strategies can affect latency performance [14]. If LEO satellites allocate more resources, then communication service requirements are better satisfied, improving latency performance and lowering the resource price. If the allocated resources for mobile users decrease, then latency performance will be worse, which raises the resource price. When service resources increase, LEO satellites should increase the service price. The relationship between the service price and allocated resources is expressed as where δ i , α i , and β i are weighted parameters.
Spectrum resources are unavailable when LEO satellites are in a NLEO satellite coverage area, and mobile users are placed in a queue. Assuming the arrival rate of a NLEO satellite is λ, following a Poisson distribution, then the probability that the LEO satellites are in a NLEO satellite coverage area is The objective of each LEO satellite is to maximize the revenue earned from mobile users, subject to The resource pricing and allocation problem in the multilayer satellite system is formulated as a dynamic game, as follows: • The LEO satellites are the players.
• The system state is the resource price.
• The strategy of each LEO satellite consists of the allocated resources for mobile users.

Game Analysis
The optimal strategies for the proposed problem are now discussed. Based on the system model and problem formulation, a dynamic resource pricing and allocation model is provided, as shown in Fig. 2.
The Bellman dynamic programming technique is used to solve the proposed dynamic game model. As the resource-pricing strategy is formulated as the system state and the resource allocation strategy is the control variable, the optimal pricing strategy is achieved once the optimal resource allocation strategy for each LEO satellite is obtained. The following definitions must be developed before obtaining the optimal strategies.

Definition 1
The resource allocation strategy r i * (t) is the optimal strategy for each LEO satellite, which, for all strategies r i (t) = r i * (t), satisfies To obtain the resource allocation strategy for each LEO satellite, a Hamiltonian function is constructed for the proposed dynamic game model. This is a key component of the Bellman dynamic programming technique [15], and is defined as follows.

Definition 2 The Hamiltonian function of LEO satellite i is
where i (t) is the co-state function, which satisfies From the Hamiltonian function above, the first derivative is calculated to find the optimal resource allocation strategy for each LEO satellite, as explained by the following theorem.

Theorem 1
The optimal resource allocation strategy r i * (t) is obtained based on the open-loop Nash equilibrium of the proposed dynamic game in (5) and (6), where p i * (t) is the corresponding optimal resource pricing strategy if there is a constant function i (t) for LEO satellite i that satisfies Considering the optimal resource allocation and pricing problem given by (3) and (4), based on Pontryagin's maximum principle, the Nash equilibrium solution is achieved for each LEO satellite, as given in the following theorem.
Theorem 2 There is a unique open-loop Nash equilibrium for each LEO satellite in the resource pricing and allocation problem, with optimal resource allocation strategy where [p i (t) , i (t)] are the solutions to the following equations: Proof: The partial derivative of the Hamiltonian function given in (8) is calculated, giving Setting the partial derivative to zero, where r i * (t) is the optimal resource allocation strategy for LEO satellite i. i (t) is the co-state function, which satisfies Taking the optimal resource allocation strategy of LEO satellite i into the various functions of the service price, The algorithm to obtain the optimal resource allocation strategy for a LEO satellite is expressed as Algorithm 1.  (10) and (11); 9. end 10. for LEO satellite i do 11. Take strategy r i * (t) into price dynamic function in formula (6) and co-state function in formula (9); 12. end 13. for LEO satellite i do 14. Calculate differential equations according to formulas (12) and (13) 14. end 15. for LEO satellite i do 16. Take differential equations back to strategy r * i (t); 17. end 18. return strategy r i * (t) Based on the proposed algorithm and the Nash equilibrium solutions of the proposed dynamic game, each satellite controls the resource allocation strategy and optimizes the objective function. An optimal pricing strategy is obtained for each satellite. As the differential gamebased model is non-cooperative, satellites do not require cooperation. To optimize their objective functions, the satellites non-cooperatively control their resource allocation strategies.

Numerical Simulations
Numerical simulations of the optimal pricing strategies and resource allocation strategies of LEO satellites were conducted using one NLEO satellite, which is the authorized user of spectrum resources, and three LEO satellites. When the LEO satellites are in the coverage area of the NLEO satellite, their spectrum resources are retrieved. Tab. 1 provides parameters for the simulations.  3 provides the optimal resource allocation strategies for LEO satellites when the arrival rate of the NLEO satellite is 0.15. The LEO satellites allocate more resources at the beginning of the game to attract mobile users and provide satisfactory service. After approximately five iterations, each LEO satellite decreases the allocated resources to reduce the interference caused by the NLEO satellite. When the resource allocation strategies of the LEO satellites converge, LEO-2 has the most allocated resources and LEO-1 has the least; however, LEO-1 has the fastest convergence rate. Before the resource allocation strategies converge, LEO-3 has the largest value of allocated resources. The LEO satellite's optimal resource allocation strategies when the arrival rate of the NLEO satellite is changed were also simulated, as shown in Figs. 4a and 4b. The arrival rates of the NLEO satellite in Figs. 4a and 4b are 0.5 and 0.85, respectively. It is shown that the LEO satellites increase the allocated resources at the beginning of the game. After five iterations, the LEO satellites decrease their allocated resources. Comparing the resource allocation strategies in Figs. 4a and 4b with that in Fig. 3, it is observed that the allocated resources decrease with the increase of the NLEO satellite's arrival rate.  5 shows the optimal resource-pricing strategies for LEO satellites when the arrival rate of the NLEO satellite is 0.15. The resource-pricing strategy converges to a stable value after five iterations, which is fast. Based on the resource allocation strategies, LEO-2 and LEO-3 will increase their resource price and LEO-1 will decrease its price. LEO-3 has the highest resource price and LEO-1 has the lowest.

Conclusions
The resource allocation problem of a multilayer satellite system was investigated and a dynamic game-based resource pricing and allocation model, using differential equations to formulate the resource-pricing strategy of each LEO satellite, proposed. Utility maximization based on an objective function was proposed for each LEO satellite, with the resource allocation strategy as the control variable. Bellman dynamic programming was used to maximize the objective function, and the Nash equilibrium solution of resource allocation of each LEO satellite was obtained. An algorithm was developed for the resource pricing and allocation model, and numerical simulations demonstrated its accuracy.

Conflicts of Interest:
The authors declare that they have no conflicts of interest to report regarding the present study.