A game theoretic model of the behavioural gaming that takes place at the EMS - ED interface (cid:2)

This research describes the development and application of a 3-player game theoretic model between two queueing systems and a service that distributes individuals to them. The resultant model is used to explore dynamics between all players. The ﬁrst aspect of this work is the development of a queueing system with two consecutive waiting spaces where the strategic managerial behaviour corresponds to how individuals use these waiting spaces. Two modelling techniques are deployed: discrete event simulation and Markov chains. The state probabilities of the Markov chain system are used to extract the performance measures of the queueing model (e.g. mean time in each waiting room, mean number of individuals in each room, etc.). A 3-player game theoretic model is subsequently proposed between the two queueing systems and the service that distributes individuals to them. In particular this can be viewed as a 2-player normal-form game where the utilities are determined by a third player with its own strategies and objectives. A backwards induction technique is used to get the utilities of the normal-form game between the two queueing systems. This particular system has many applications, including those in healthcare where it captures the emergent behaviour between the Emergency Medical Service (EMS) and the Emergency Department (ED). The impact of time-target measures on patient well-being is explored in this paper. © 2022 The Authors. Published by Elsevier B.V. This is an open access article under the CC BY license ( http://creativecommons.org/licenses/by/4.0/ )


Introduction
Emergency departments (EDs) are under increasing pressure to meet patient waiting time targets and satisfy regulations ( Halliwell, 2021 ).It is widely reported (e.g.Knapper, 2021 ;Lemmer, 2021 ;Mahase, 2020 ) that ED congestion severely impacts not only patients in the ED but also Emergency Medical Services (EMS).A major concern for ambulances is that they are held waiting parked outside the ED to offload (dispatch) their patient when the ED is particularly busy ( Clarey, Allen, Brace-McDonnell & Cooke, 2014 ).Since the patient waiting time in ED is measured from the time they enter the ED itself, there is no incentive, should the patient be stable in the ambulance, to offload them from EMS to ED services.As a result, ambulance blocking not only impacts on patients waiting for ED service, but has a major knock-on effect to delaying the ability of ambulances to respond to new EMS calls, thus placing lives at risk ( Day, 2021 ).
There are numerous news articles that focus on the complexity that arises when ambulances stay blocked outside of the hospital for a long amount of time ( Aitken, 2021 ;Crouch, 2021 ).Some news reports comment on the long idle time of ambulances when they are not in use ( Thomas, 2021 ) and there are several reports of examples where this became an issue for new patients ( McAdams, 2021 ) and paramedics ( Clarke, 2021 ).
This paper aims to describe the EMS-ED interface using a game theoretic model informed by an underlying queueing model.The model describes the situation where an ambulance service would have to distribute its patients between two EDs.The two EDs can be thought of as two queueing systems and the EMS as a distributor that distributes patients to them, aiming to minimise some performance measure.The patients that are distributed by the EMS arrive at the hospital via an ambulance and are then either offloaded at the ED or stay blocked outside in the ambulance.Whether or not the ambulance and its patient stay blocked is determined by the threshold that the given ED chooses to play.A high threshold indicates that the ED accepts ambulance patients even if it is relatively full, while a low threshold means that the ED blocks ambulances more frequently.
In the United Kingdom, the National Heath Services (NHS) sets some regulations on ED performance.One of these regulations is that 95% of patients that arrive at the ED should be admitted, transferred or discharged within four hours.This is where gaming behaviour might be observed between the EDs and the EMS.An assumption of this work is that some managerial decision making is involved in choosing when to start blocking ambulances.This is similar to Deo and Gurvich (2011) .
The major contributions of this paper are: • A queueing model with two consecutive waiting spaces where one would serve as a parking space for the ambulances.
• Analytic performance measure formulas for the queueing model.• A 3-player game theoretic model between the EMS and two EDs.
• Numerical experiments showing emergent behaviour of gaming between EDs and the EMS.
Specifically, our focus is on the construction of a 3-player game theoretic model between two queueing systems and a service that distributes individuals to them.The resultant model is then used to explore the emergent dynamics between the three players.This study explores two new concepts: getting performance measures for a new queueing theoretic model with a parking space and a service centre, and using a learning algorithm to model the emergence of behaviour.The developed theoretical model is illustrated through the application to a healthcare system of two EDs and the EMS, exploring the inefficiencies that emerge and ways to apply some incentive mechanisms to improve them.The EDs are modelled as two queueing systems each with a tandem buffer and a service centre.The performance measures are then used as the utilities of the game.The novelty of the queueing model here is a contribution not only the game theoretic literature but also to the queueing theoretic literature.To the authors knowledge, no such model of a tandem queueing model with a pair of parameters for the buffer has been previously considered.
This paper consists of two main sections.Section 3 presents a novel queueing model for a hospital with two types of patients and two waiting zones.A detailed description of how to acquire the performance measure formulas of such queueing system is given.Section 4 gives an overview of the game theoretic model and several theoretic results pertaining to the performance measures of this model which are used to build the utilities of the game.

Literature review
A number of papers have been published that touch upon the use of queueing models together with game theoretic concepts.In Chen and Wan (2003) the authors study a simultaneous price competition between two firms that are modelled as two distinct queueing systems with a fixed capacity and a combined arrival rate.They calculate the Nash equilibrium both for identical and heterogeneous firms and show that for the former a pure Nash equilibrium always exist and for the latter a unique equilibrium exists where only one firm operates.The authors have also extended their model in Chen and Wan (2005) by allowing the players (firms) to choose capacities.A main result from this paper was that when both firms operate independently as a monopoly, the equilibria are socially optimal, but this is not the case when the firms operate together.Another extension of Chen and Wan (2003) was introduced in Cheng, Demirkan and Koehler (2003) where a longrun version of the competition was considered that also had capacity as a decision variable.An additional paper that focuses on competition is ( Fan, Kumar & Whinston, 2009 ) where the authors created a competition between two sellers where seller 1 supplies a product instantly and seller 2 is modelled as a make-to-order M/M/1 queue.The game that is played requires the two sellers to make a choice on the price of the product and then seller 2 to set a capacity that guarantees a maximum expected delay.In our work, while giving some consideration to equilibrium behaviour, similar to the work of Chen and Wan (20 03,20 05) , emergent behaviour is more precisely addressed by considering learning algorithms like asymmetric replicator dynamics ( Fudenberg, Drew, Levine & Levine, 1998 ).
In the above models, the players are attempting to increase their share of individuals choosing to queue.In public healthcare type settings, this is not necessarily the case.Rational usage of public services will not necessarily lead to a socially optimal outcome.Rather, the overall service needs to be considered as players aim to minimise their experienced congestion.In Sadat, Abouee-Mehrizi and Carter (2015) a healthcare application was studied where patients could choose between two hospitals, where a utility function is derived that is based on patients' perceived quality of life.In Knight and Harper (2013) the authors place the individuals' choices between different public services within the formulation of routing games and measure inefficiencies using a concept known as the price of anarchy (PoA) ( Koutsoupias & Papadimitriou, 1999 ).They show that the price of anarchy increases with worth of service and that is low for systems with insufficient capacities.In Chen, Zhang and Chen (2020) a two-tier healthcare system with a capacity constrained is studied where patients can choose between two systems to receive their service.The first system is labelled as the free system (public government-funded hospital) which offers service without seeking any profit and the second one is the toll system (private hospital) that aims to maximise its own profit.The authors, also compare the two-tier system with its one-tier equivalent, where only the free system exists.In Knight, Komenda and Griffiths (2017) a normal form game is built that is informed by a two-dimensional Markov chain in order to model interactions between critical care units.In Wang, Wang, Zhang and Wang (2021) a queueing-game-theoretical model is introduced where there are two types of service providers; a high quality high-congested hospital and a low quality low-congested hospital.The authors study a two-stage Stackelberg game where the government is the leader and the arriving patients are the followers .In Deo and Gurvich (2011) the authors study the network effect of ambulance diversion by proposing a non-cooperative game between two EDs that are modelled as a queueing network.Each ED's objective is to minimise its own waiting time and chooses a diversion threshold based on the patients it has.In equilibrium both EDs choose to divert ambulances in order to avoid getting arrivals from the other ED.In this paper this concept is extended by allowing the ambulance service to decide how to distribute its patients among the two EDs.The players of the game are both the hospitals and the customers of the hospitals, as opposed to the previous models which are one or the other.Thus, the novelty of our work is combining both these aspects.
Another specific part of our research, as described later in the paper, is the construction of a queueing system with a tandem buffer and a single service centre.There are several examples from literature that touch upon queueing models with tandem queues.In D' Auria and Kanta (2015) the authors explore threshold joining strategies in a Markov model that has two tandem queues.Another example is the one described in Burnetas (2013) where they investigated a network of multiple tandem queues where customers decide which queue to attend before joining.Similarly, in Ba ¸s ar and Srikant (2002) the authors examine a network of N tandem M/M/1 queues and with multi-type customers.The customers in this paper react to a price p by picking demand rates that maximise utility.In Veltman and Hassin (2005) a profit maximisation problem is studied that has two servers; an M/M/1 queue and a parking service providing complementary service while the customer is in the first service.The providers gain a reward when customers com- plete both services and no reward when they finish one of them.One of the main conclusions of this study is that by increasing the general demand both providers lower their prices to compensate for the increase in wait.The problem was later extended by Sun, Li, Tian and Zhang (2009) where they considered arrivals of batches that can share the parking service.Finally, ( Afeche, 2007 ) examines a tandem network of two M/M/1 queues that are ran by two different profit-maximising service providers.The network receives three types of customers; those requiring both services, customers requiring the first service and customers requiring the second service.The authors showed that optimal prices also maximise social utility and that removing two types of customers that don't need both services leads to higher profit and lower demand rate.In our work, the concepts described in Ba ¸s ar and Srikant (2002) ; Burnetas (2013) ; D' Auria and Kanta (2015) are extended by introducing a threshold parameter that determines when individuals can progress from one queue to the other.

A queueing model for the ED-EMS interface
In this section, a more in-depth explanation of the queueing model shown in Fig. 1 will be given.This is a queueing model that consists of two waiting zones: the parking space and the hospital waiting space.
The model consists of two types of individuals; type 1 and type 2. Type 2 individuals are patients arriving in ambulances who can be blocked (usually patients that are deemed not to be critical) and type 1 individuals are individuals arriving from other sources (e.g walk-in patients, urgent patients from either walk-ins or ambulances).Type 1 individuals arrive instantly at the hospital's waiting space and wait to receive their service.Type 2 individuals arrive at the parking space and wait there until they are allowed to move into the hospital.They are allowed to proceed only when the number of patients in the hospital is less than the pre-determined threshold T .When the number of individuals is equal to or exceeds this threshold, all type 2 patients that arrive will stay blocked in the parking space until the number of patients in the hospital falls below T .This is shown diagrammatically in Fig. 1 .The parameters of the described queueing model are: The service rate for individuals receiving service • C: The number of servers (either healthcare professionals or available beds in the ED) • T : The threshold at which type 2 individuals are blocked Under the assumption that all rates (arrival and service) are Markovian the queueing system corresponds to a Markov chain ( Kemeny & Snell, 1976 ).The states of the Markov chain are denoted by (u, v ) where: • u is the number of individuals blocked in the parking space • v is the number of individuals waiting or being served in the hospital We denote the state space of the Markov chain as S = S(T ) which can be written as the disjoint union (1) .

S(T )
The generator matrix Q of the Markov chain consists of the rates between the numerous states of the model.Every entry represents the rate from state i = (u i , v i ) to state j = (u j , v j ) for all (u i , v i ) , (u j , v j ) ∈ S. The entries of Q can be calculated using the state-mapping function described in (2) : Note that here denotes the overall arrival rate in the model by both types of individuals (i.e. = λ 1 + λ 2 ).A visualisation of how the transition rates relate to the states of the model can be seen in the general Markov chain model shown in Fig. 2 .
In order to consider this model numerically an adjustment needs to be made.The problem defined above assumes no upper boundary to the number of patients that can wait in the hospital or to the ones that are blocked in the parking space.Therefore, a different state space ˜ S is constructed where ˜ S ⊆ S and there is a maximum allowed number of patients N that can be in the hospital and a maximum allowed number of ambulances M that can be blocked in the parking space: The generator matrix Q defined in (2) can be used to get the probability vector π .The vector π is commonly used to study stochastic systems and it's main purpose is to keep track of the probability of being at any given state of the system.π i is the steady state probability of being in state (u i , v i ) ∈ ˜ S which is the i th state of ˜ S for some ordering of ˜ S .The term steady state refers to the instance of the vector π where the probabilities of being at any state become stable over time.Thus, by considering the steady state vector π the relationship between it and Q is given by: dπ dt Using vector π there are numerous performance measures of the model that can be calculated.The following equations utilise π to get performance measures for the average number of patients at the different nodes of the queueing model: • Average number of patients in the entire system: • Average number of patients in the hospital: • Average number of patients/ambulances in the parking space: Consequently, there are some additional performance measures of interest that are not as straightforward to calculate.Such performance measures are the mean waiting time in the system (for both type 1 and type 2 individuals), the mean time blocked in the parking space (only valid for type 2 individuals) and the proportion of individuals in the hospital whose waiting time falls within a predefined time target (for both types).

Waiting time
Waiting time is the amount of time that patients wait in the hospital's waiting space before they can receive their service.For a given set of parameters there are three different performance measures around the mean waiting time that can be calculated.The mean waiting time of type 1 individuals: The mean waiting time of type 2 individuals: The overall mean waiting time: Here S (1) A and S (2) A are the set of accepting states for type 1 and type 2 individuals.These are the set of states that the model is able to accept a specific type of individuals.

S (1)
Eq. 6 makes use of the proportion of type 1 and type 2 individuals that are not lost to the system.These probabilities are given by P L 1 and P L 2 where: Appendix B gives more details on the recursive formula that Eqs. ( 4) , ( 5) and (6) originate from.
Fig. 3 shows a comparison between the calculated mean waiting time using Markov chains and the simulated waiting time using discrete event simulation over a range of values of λ 2 (details of the discrete event simulation model are given in appendix Appendix A ).The figure is used to demonstrate the accuracy of the waiting time formula of the constructed queueing model as well as the effect of truncating the model.The simulation was run 100 times and the recorded mean waiting time at each iteration is used to populate the violin plots.In detail, Fig. 3 shows the calculated mean waiting time using the Markov chain, using a truncated simulation and using a simulation with infinite capacity (without the artificial parameters N and M).Each plot corresponds to different values of N and M and is run over different values of λ 2 .The untruncated simulation values are the same at all three graphs since the effect of truncation does not apply to it.The waiting times generated by the truncated simulation match the ones generated by the Markov chains model.Note that this comparison includes both type 1 and type 2 individuals.A separate comparison of only type 1 and only type 2 individuals can be found in appendix Appendix E .

Blocking time
Blocking time is the amount of time that type 2 patients wait in the parking space before they are allowed to proceed into the hospital.Unlike the waiting time, the blocking time is only calculated for type 2 individuals.That is because type 1 individuals cannot be blocked.Thus, one only needs to consider the pathway of type 2 individuals to get the mean blocking time of the system.The mean blocking time can by calculated using: Here S (2)  A is the set of accepting states of type 2 individuals (defined in Eq. ( 8) ) and A i (u, v ) for i ∈ { 1 , 2 } is the state that the system would go to when the system is at state (u, v ) and an individ- The term b(u, v ) is the mean time that an individual will be blocked for, when the individual arrives in the system at state (u, v ) .For all the states of the system b(u, v ) is given by: Note that S b is defined as the set of states where individuals can be blocked and is given by: Additionally, c(u, v ) is the mean sojourn time for each state and p s and p a are the probabilities that the next event to occur will be a service completion or an arrival of a type 1 individual: The system of equations produced by ( 13) can be solved by considering the linear system Zx = y .Assuming i and j represent states Eq. ( 18) shows this.
Additional details on the blocking time formula (10) can be found in appendix Appendix C .
Fig. 4 illustrates a comparison between the formulas that arise from the Markov chain model and the equivalent values of the blocking time extracted from discrete event simulation (appendix Appendix A ).The blocking time is calculated using both methods for a range of values of λ 2 .The figure is used to demonstrate the accuracy of the blocking time formula of the constructed queueing model as well as the effect of truncating the model.The simulation was run 100 times and the recorded mean blocking time at each iteration is used to populate the violin plots.Similar to Fig. 3 , these plots shows a comparison between the calculated mean blocking time using Markov chain, using a truncated simulation and using a simulation without the artificial parameters N and M. The blocking times generated by the truncated simulation match the ones generated by the Markov chains model.Note that this comparison includes only type 2 individuals since type 1 individuals cannot be blocked.

Proportion of individuals within target
Another performance measure that is taken into consideration is the proportion of individuals whose time in the hospital (waiting and service time) is within a specified time target t.Similar to Section 3.1 , three formulas are needed for this performance measure.
The proportion of type 1 individuals within a time target: The proportion of type 2 individuals within a time target: The terms A 1 (u, v ) and A 2 (u, v ) are defined by Eqs.11 and 12 in Section 3.2 .The overall proportion individuals within a time target (where P L 1 and P L 1 are defined in (9) ): Here P (X (1) (u, v ) ) and P (X (2) (u, v ) ) are defined as the proportion of individuals within the time target t when starting from state (u, v ) .These expression can be calculated by: where r = (v − C, 1) , λ = (Cμ, μ) and λ 0 = 0 , r 0 = 1.
The function k,l used in Eqs. ( 22) and ( 23) is defined as: Please refer to Appendix D for a more in-depth explanation of the origins of equations ( 19) -( 23) .
Fig. 5 shows a comparison of the mean proportion of individuals within target when using Markov chains and discrete event simulation (appendix Appendix A ).The figure is used to demonstrate the accuracy of the formula for the proportion of individuals within time of the constructed queueing model as well as the effect that truncating the model has on the formula.The simulation was run 100 times and the recorded proportions at each iteration is used to populate the violin plots.Similar to Figs. 3 and 4 , these plots shows a comparison between the calculated mean proportion of individuals within time using Markov chain, using a truncated simulation and using a simulation without the artificial parameters N and M. The proportions generated by the truncated simulation match the ones generated by the Markov chains model.Note that this comparison includes both type 1 and type 2 individuals.A separate comparison of only type 1 and only type 2 individuals can be found in appendix Appendix E .

Truncation effect timings
The choice of the artificial parameters N and M is an important decision of the model.In the untruncated simulation these values are not needed.This is not possible when obtaining the steady state probabilities of the finite state Markov chain.Table 1 shows the relative timings of the different approaches used to get the performance measures.

Strategic manipulation of the ED-EMS interface
The problem studied is a 3-player normal form game.The players are: • the decision makers of two Emergency Departments (EDs) • the Emergency Medical Services (EMS) that distribute individuals in ambulances to the EDs This is a standard normal form game ( Maschler, Solan & Zamir, 2013 ), in that each player in this game has their own objectives which they aim to optimise.More specifically, the EDs' objective is captured by an upper bound of the time that a fixed proportion of individuals spend in the system, while the EMS aims to minimise the time that its ambulances are blocked.This can be generalised for any such system where instead of EDs there are some queueing systems and instead of the EMS there is some distributor that allocates individuals to the queueing systems.
The parameters of the model correspond to the following parameters of the ED and the EMS: • λ 2 : The rate of patients (who can be blocked) that the EMS receives and distributes to EDs The number of available resources (healthcare professionals) in the ED i ∈ { A, B } • T i : The action that ED i ∈ { A, B } chooses to play which corresponds to the threshold at which they do not accept EMS patients.
• N i : The total patient capacity of the ED i ∈ { A, B } • M i : The total parking capacity of the ED i ∈ { A, B } • t: The time target for both EDs • α ∈ [0 , 1] : Weighted average of blocking time and lost individuals ( Eq. 25 )  The strategies of the two EDs are the range of thresholds that they can choose from and their utilities are the proportions of individuals whose time in the system is within a predetermined target time.The EMS has to decide how to distribute its patients among the two EDs so that the weighted combination of the ambulance blocking time and the percentage of lost ambulances is minimised.This can be illustrated by Fig. 6 .The interaction between the two EDs is a normal form game that is then used to inform the decision of the EMS.Note that the formulated game here assumes that prior to making a choice the EMS knows the strategies that each ED is playing ( Fig. 7 ).This corresponds to reacting to experienced delays.
The queueing systems of the hospitals are designed in such a way where they can accept two types of individuals ( Section 3 ).Each hospital may then choose to block type 2 individuals when the hospital reaches a certain capacity.The strategy sets for each hospital is the set { T ∈ N | 1 ≤ T ≤ N} where N ∈ { N A , N B } are the total capacities of hospitals A and B .We denote the chosen actions from the strategy set as T A , T B and call these threshold s.
Both hospitals follow a queueing model with two waiting spaces for individuals.The first waiting space (i.e. the waiting space of the hospital) is where the patients queue right before receiving their service and has a queue capacity of N − C, where N is the total capacity of the hospital and C is the number of healthcare professionals able to see them.The second waiting space (i.e. the parking space for ambulances) is where ambulances, that are sent from the EMS distributor, stay until their patients are allowed to enter the hospital.The parking space has a capacity of M and no servers.This is shown diagrammatically in Fig. 1 .
Note here that both types of individuals can become lost to the system.An individual allocated from the ambulance service becomes lost to the system whenever an arrival occurs and the parking space is at full capacity ( M ambulances already parked).Similarly, type 1 individuals get lost whenever they arrive at the waiting space of the hospital and it is at full capacity ( N − C individuals already waiting).
Following this queueing model, the two queueing systems' choice of strategy will then rely solely on satisfying their own objective.The objective function is defined as: arg max where W is the waiting time of a potential individual, t is the time target and ˆ P is the percentage of individuals need to be within that target.In other words, their aim is to find the threshold that minimises the difference between the probability P (W i < t ) and the percentage goal (or maximise its negation).
The third player, the ambulance service, has their own choices to make and their own goals to satisfy.The strategy set of the third player is the proportion 0 ≤ p A ≤ 1 of individuals to send to hospital A. Similarly the proportion of individuals to send to hospital B is given by p B = 1 − p A .In addition, the ambulance service aims to minimise any potential blockages that may occur, given the pair of thresholds chosen by the two hospitals.Thus, its objective is to minimise the blocked time of the individuals ( B A and B B ) that they send to hospitals A and B .Apart from the time being blocked, an additional aspect that may affect the decision of the distributor is the proportion of lost individuals L A and L B .Eq. 25 can be used to capture a mixture between the two objectives L i and B i where Here, α represents the "importance" of each objective, where a high α indicates a higher weight on the proportion of lost individuals and a smaller α a higher weight on the time blocked.The choice of p A and p B rely solely on Eq. 25 .Note that the current system is modelled using unobservable queues which is not necessarily an unrealistic approach ( Shone, Knight & Williams, 2013 ).Another approach would be to allow the ambulance service to observe the state of each hospital before deciding which one to join for a given individual.There are several other factors that can affect the routing of the patients that are outside the scope of this paper.For example the distance from each hospital or even the priority level of each patient may be a significant factor that affects the ambulance service's decision.
Using either Eq. ( 25) or ( 24) gives an imperfect information extensive form game.An imperfect information game is defined as an extensive form game where some of the information about the game state is hidden for at least one of the players ( Berwanger & Doyen, 2008 ).In this study the state of the problem that is hidden is the threshold that each hospital ED chooses to play.In other words, each hospital chooses to play a strategy without knowing the other hospital's strategy.The ambulance service then, fully aware of the chosen threshold strategies, distributes individuals among the two systems in order to minimise the time that its ambulances will be blocked.Fig. 7 illustrates this.
Hospital H A decides on a threshold, then the hospital H B chooses its own threshold, without knowing the strategy of H A , and finally the ambulance service makes its choice.Note here that the dotted line represents the fact that H B is unaware of the state of the game when making its own decisions.
From Eq. 24 the utilities of the hospitals can be formulated.The 2-player normal form game between the two hospitals is defined by: • Players : Hospitals H A and H B • Strategy spaces : Consequently, the payoff matrices of the game can be populated by these utilities: Based on the choice of strategy of these two hospitals, the ambulance service will then make their own choice of the proportion of individuals to send to each system.

Building the game
The problem defined in this section describes a normal-form game between the decision makers of two hospitals and a third player, the ambulance service, that decides how to distribute individuals to the two systems.The strategy space of the two hospitals is defined as the possible values that the threshold parameter can take T i ∈ [1 , N i ] .Then, the ambulance service has to decide on the proportion of individuals to send to each hospital p A and p B , where p A , p B ∈ [0 , 1] and p A + p B = 1 .In practice this would correspond to a learned behaviour through experience of waiting at each hospital.Fig. 7 shows a diagrammatic representation of the game to be played and the decisions to be made.As described in Section 4 , hospital A decides on a strategy and at the same time, hospital B chooses its own threshold but unaware of the first hospital's choice.Finally, the ambulance service makes its choice based on the strategies that the hospitals chose to play.
The utilities to each player can be represented by 3 matrices: the two payoff matrices of the normal form game and the routing matrix.The payoff matrices and their utilities are defined by Eqs. ( 26) and ( 27) .
The routing matrix holds the values (p A , p B ) which are the proportion of ambulance patients to send to queueing systems A and B .Each pair (p A , p B ) can be calculated using Eq. ( 25) and is essentially a best response to the actions of the hospitals.Thus, using Eq. ( 25) for all possible sets of thresholds, we can get the full routing matrix R ( Eq. 28 ) that consists of the proportions to send to hospital A ( p A ) and to hospital B ( p B ).
Note that since p A i, j + p B i, j = 1 the routing matrix needs only to store one of the two values; either p A i, j or p B i, j .Thus, the routing matrix R can be simplified to: The game can thus be partitioned into a normal form game between the two hospitals and then finding the ambulance service's best strategy.Now consider Fig. 7 and the flow of the game that was described (i.e.H A , H B → D ).Due to the fact that the payoff matrices A and B depend on the routing matrix R the entries of the matrices are calculated in a backwards way ( D → H A , H B ).This is done using backwards induction.For each action choice of the hospitals, first solve the game from the ambulance's point of view.This in effect results in a 2-player normal form game representing the hospital's point of view.Thus, for every pair of strategies T A , T B , the values of p A and p B that satisfy Eq. ( 25) are found numerically using Brent's bisection algorithm ( Brent, 1973 ).Each pair (p A , p B ) corresponds to the best response of the ambulance service to the two hospitals' played strategies.Finally, using the routing matrix, Eq. ( 26) can also be used to populate the payoff matrices of the hospitals since we now know the arrival rate of each hospital.
Having calculated the payoff matrices A and B , several algorithms can be used to measure some form of the emergent behaviour.One possibility would be to compute the Nash equilibrium which is the point of the game were both players have no incentive to deviate from their played strategies ( Kreps, 1989 ).In other words their chosen strategies are a best response to each other.
Computation of Nash equilibria can be done in a relatively efficient way using the Lemke Howson algorithm ( Lemke & Howson, 1964 ).Lemke-Howson uses best response polytopes to get one of the Nash equilibrium of the game.Other algorithms exists that will compute all Nash equilibria but for large games the computational complexity becomes problematic.All game theoretic calculations were done in Python using the Nashpy library ( The Nashpy project developers, 2021 ).
Another approach to measuring emergent behaviour is to consider the emergence itself and not only stable end points.Indeed, some Nash equilibria might not arise naturally.Thus in order to analyse the strategies played by the two hospitals, the learning algorithm asymmetric replicator dynamics is used ( Accinelli & Carrera, 2011 ).The two hospitals are modelled as two separate populations where each individual in the population is assigned a strategy.As the game progresses the proportion of each player playing each strategy changes based on their previous interactions.The fitness of each strategy is defined as: Here, x ∈ R m ×1 and y ∈ R n ×1 correspond to the proportion of individuals that play each strategy in each population.Similarly, the average fitness of each strategy is given by: The rate of change of strategy i of both types of individuals is captured by: In addition to asymmetric replicator dynamics, the learning algorithms fictitious play and stochastic fictitious play ( Fudenberg et al., 1998 ) were used.

Results
This subsection aims to analyse how the gaming framework can affect the performance measures of the two hospitals and how to escape certain inefficient situations.
The most commonly used method for analysing normal form games is the Nash equilibrium described in Section 4.1 .Consider the following game: The set of possible actions to choose from for player 1 and player 2 is the set of thresholds that the EDs can choose from: For this example, the only Nash equilibrium of the game is achieved when both players choose a threshold of T A = 10 , T B = 10 .This means that the two players' best response to each other is to only block ambulances when they are full.
Nash equilibria is a theoretical measure which can be inconsistent with intuitive notions about what should be the outcome of a game ( Myerson, 1978 ).Therefore it might not be the best way to describe human behaviour.Since the work of Maynard Smith ( Smith, 1986 ), evolutionary game theory gives the tools for the emergence of stable behaviour.One such model that allows for asymmetric payoffs, as is the case above, is replicator dynamics described in section (4.1) .Stable outcomes of this algorithm will correspond to a subset of Nash equilibria but more importantly, will give a model of emergent behaviour.
The use of a learning algorithm allows to investigate, not only the outcome of the game, but also how that outcome is reached.Consider Fig. 8 .By running asymmetric replicator dynamics on the system the behaviour that emerges can be observed.It can be seen that for this particular set of parameters the strategies of the two hospitals converge over time.Both hospital 1 (row player) and hospital 2 (column player) seem to be playing the same strategy s 10 which indicates that thresholds T A = 10 and T B = 10 are played.What is more important in this example is how the two hospitals reached these decisions which also highlights the importance of using a learning algorithm.Hospital 2 is able to reach the decision in a short amount of time while hospital 1 takes longer and goes through numerous strategies to get there.
In order to analyse how efficient the strategies played at every iteration are, the concept of the price of anarchy is used.Price of anarchy (PoA) is a measure that is used to quantify the efficiency of a behaviour ( Roughgarden, 2005 ).In other words the price of anarchy is the worst-case equilibria measure and it is defined as: Here, S is the set of all sets of strategies (s A , s B ) , E is the set of all possible sets of equilibria and F is the cost function to measure the efficiency for.
To study the efficiency of the strategies being played, a new concept is introduced that considers the ratio between each hospital's best achievable blocking time and the one that is being played.This new concept is defined as the compartmentalised price of anarchy of the players of the game and is defined as PoA i ( ˜ s ) , where i ∈ { A, B } to distinguish among the two players/hospitals where ˜ s is the strategy that is being played by player i .The compartmentalised price of anarchy aims to measure inefficiencies in the model.The PoA for the blocking time of player i for strategy ˜ s is given by: For this particular scenario, two busy queueing systems will be used with a high traffic intensity ( ρ > 1 ).Consider a game with two smaller (lower N i , M i ) and busier (higher λ 1 i and λ 2 ) hospitals with the following set of parameters:  Initial scenario: Using Eq. ( 36) and asymmetric replicator dynamics, the emergent behaviour can be measured and the compartmentalised price of anarchy at every iteration for both players.Fig. 9 shows the strategies that are being played and the values of PoA A (s ) and PoA B (s ) for all iterations of the learning algorithm until it reaches an evolutionary stable pair of strategies.
The learning algorithm reaches a stable pair of strategies where T A = 5 and T B = 6 .Thereafter, the price of anarchy for both players stabilises and barely increases.
Increasing λ 2 : Fig. 10 shows a similar run of the algorithm but when the strategies begin to stabilise, an increase in the arrival rate of ambulances occurs (i.e.λ 2 = 24 ).
By increasing λ 2 there is no change as to how players behave ( T A = 5 , T B = 6 ), but the efficiency of the system does change.There is a decline in the price of anarchy of the blocking time which at first glance indicates that upon flooding the system it becomes the loss in efficiency due to rational individual behaviour decreases.This is non-sensical however.What it really shows is that the steep increase in λ 2 leaves the system unable to cope regardless of the decisions made.
Increasing number of servers C A and C B : Fig. 11 shows a run of asymmetric replicator dynamics with a change in the number of servers of the hospitals.The number of servers are increased from In this case, both the behaviour as well as the price of anarchy change.The players change their strategies from T A = 5 , T B = 6 to T A = 6 , T B = 7 and the PoA of the blocking time goes down.By adding more resources to the models they are able to increase their efficiency.Although this is a good way to escape such inefficiencies, it might not always be cost efficient.
Incentivising players: From Figs. 10 and 11 it is observed that we can change some parameters of the model to make it more efficient.The approach used on Fig. 12 is slightly different than the previous cases.Once the played strategies in asymmetric replicator dynamics strategies converge, the payoff matrices of the two are scaled in such a way so that the utilities of the selected strategy are penalised.This corresponds to a precise policy change where more societally beneficial behaviours are incentivised.
Matrices A and B represent the original payoff matrices while matrices ˜ A and ˜ B represent the incentivised payoff matrices.It can be observed that matrix ˜ A is a scaled version of matrix A only on the row that is most frequently played and similarly matrix ˜ B of matrix B only on the column that is most frequently played (matrix A : row 5, matrix B : column 6, see Fig. 12 ).Note that for the presentation of data, an affine transformation has been applied to obtain the values of the payoff matrices ( A i j = 10 0 0 0(a i j − 0 .999) and B i j = 10 0 0 0(b i j − 0 .999) where a i j and b i j are the raw utilities).The results are not affected by this scaling.Fig. 12 shows that players start playing strategies T A = 5 and T B = 6 and mid-run of the learning algorithm a penalty is applied to these strategies on the payoff matrix.By incentivising the players in such a way the players change their strategies to T A = 6 and T B = 7 , and thus ambulance patients are accepted in the ED more often.Hence, the PoA for both EDs is decreased, meaning that the whole system is more efficient in terms of the blocking time.

Conclusion
The motivation behind this study has been that emergency departments are under a lot of pressure to treat patients.This is, in practice, often centrally controlled through a mechanism of some sort of performance measure target.This paper shows how this can negatively impact the pathway of both the ambulance patients and the ambulance service itself.Due to some managerial decision making that takes place at the ED, ambulances stay blocked outside of the ED at the hospital's parking zone in an attempt to satisfy these regulations.The main contributions of this paper are: • A queueing model with 2 consecutive waiting spaces where one would serve as a parking space for the ambulances; • Analytic performance measure formulas for the queueing model; • A 3-player game theoretic model between the EMS and two EDs; • Numerical experiments showing emergent behaviour of gaming between EDs and the EMS.
Although our research is motivated by the particular EMS-ED example, our developed modelling framework and behavioural insights has application to similar systems across a range of sectors and settings.The queueing model can be applied to any setting where individuals may be blocked on a separate queue.An example of such setting can be any type of delivery service where customers can purchase goods either online or in-person.At busier times, the person delivering the product may be blocked outside the store in an attempt to improve the waiting times for walk-in customers.
This study explores a generic 3-player game theoretic model between the decision makers of two queueing systems and a service that distributes individuals to these two systems ( Section 4 ).
It also describes the construction of the underlying queueing theoretic model that has a tandem buffer and a single service centre ( Section 3 ).Furthermore, the formulas for the performance measures of the queueing model are also derived ( sections 3.1, 3.2, 3.3 ).This novel queueing model is the first contribution of the paper.The game theoretic model is then applied to a healthcare scenario by looking at the interface between the EDs and the EMS.The inefficiencies that emerge from the perspective of the EMS were explored along with ways to apply some incentive mechanisms to improve them.The key findings from this paper that were observed when playing the game between two EDs and the EMS are: • Inefficiencies can be learned and emerge naturally; • Targeted incentivisation of behaviours can help escape inefficiencies.
The former relates to the results of asymmetric replicator dynamics that showed that inefficient scenarios can arise by playing the game, while the latter implies that the learned inefficiencies can be escaped by carefully applying certain incentives to the players.This applied game theoretic model is the second main contribution of this paper.
The model presented here assumes the presence of only two players that can receive individuals.However, in a realistic healthcare scenario an ambulance may have to decide among multiple EDs.An immediate extension of this work would be to consider a multiplayer system that could represent a group of hospitals in a concentrated area.Additionally, the game theoretic model that was created uses a discrete strategy space for the EDs (something that is also present in various related literature ( Deo & Gurvich, 2011 ;Knight et al., 2017 )).The single threshold parameter that is used for the ED's decision may not be the best way to describe the model.In reality ED managers might adopt far more complex parameters for their decision making process.Moreover this work assumes that the EMS and EDs act in a selfish and rational way by only aiming to satisfy their own objectives.In some settings, cooperation may be observed and would therefore require an adapted modelling approach.Another extension would be to explore the behaviour of the ED staff via an agent-based simulation model.This in turn can be used to model emergent behaviour based on assumptions of individual behavioural traits of ED staff.Finally, future work could touch upon the derivation of a closed form formula for the steady state probability vector of the queueing model ( Section 3 ) to allow for faster computations of π .

Appendix A. Discrete event simulation
For the purposes of this study, a discrete event simulation (DES) model was constructed to support the Markov chain version described in Section 3 .The queueing model was built in python using the Ciw library ( The Ciw library developers, 2020 ).
The constructed model simulates a queueing system with two waiting spaces and two types of individuals.The expected behaviour of the nodes in Ciw have been modified such that individuals moving from waiting zone 2 into waiting zone 1 get blocked if there are more than T individuals in waiting zone 1.
The same performance measures described in Sections 3.1, 3.2 and 3.3 can also be calculated using the DES model.The simulation can be ran a number of times to eliminate stochasticity and the outcomes of the two methods can be directly comparable.

A1. Tutorial: Building the DES model
The DES model is constructed in a generic way that so that it can be used for any queueing system with two waiting spaces and two types of individuals.For instance, consider a queueing system with the following parameters, as described in Section 3 : This model will be studied by using ambulance_game ( Panayides, 2021 ).Install the created library in your python environment, by running the following command in the command line: Having installed the package, the following code can be used to simulate the queueing system defined earlier and get all the data records for a single run.
The above block code outputs the fourth individual record from the simulation object.The simulation object can be used to view every event that occurred in the simulation The data records can then be used to get overall performance measures about the constructed queueing model.The overall waiting time that individuals wait in waiting zone 1 can be acquired by running: This value is average waiting time of all the customers in the system for a single run.By nature, discrete event simulation can output different results for different runs of the same set of parameters.This stochasticity can be reduced by running the simulation multiple times and then getting the mean waiting time from all the runs.

A2. How-to guide A2.1. How to install:
The package can be installed by either running: in the command line or via the instructions provided in the GitHub repository.

A2.2. How to simulate the model:
The required arguments that need to be passed to the simulate_model() function are the following: • lambda_1 ( λ 1 ): The arrival rate of class 1 individuals.
• mu ( μ): The service rate of the servers.
• num_of_servers ( C): The number of servers in the system.
• threshold ( T ): The threshold that indicates when to start blocking class 2 individuals.
To get the simulation object with all the data records, the following code can be used: Additional arguments that can be passed to the function are: • system_capacity ( N): The maximum number of individuals in waiting zone 1.
• buffer_capacity M: The maximum number of individuals in waiting zone 2.
• seed_num : The seed number for the random number generator.
• runtime : How long to run the simulation for.

A2.3. How to get the performance measures for a single run:
From a single run of the simulation the data records can be used to get the average for certain performance measures.The following code can be used to get the mean waiting time, blocking time, service time and the proportion of individuals within target.

A2.4. How to get the average performance measures:
To reduce the effects of stochasticity in the simulation, the simulation can be run numerous times and get the average performance measures out of all the runs.

A2.5. How to get the steady state probabilities π :
To get the steady state probabilities of the model based on the simulation the following code can be used: Similarly to get the average steady state probabilities over multiple runs, one can use: A2.6.How to get the optimal distribution of class 2 individuals among 2 queueing models: In the scenario where there are two queueing models and a service that distributes individuals to the models, (i.e. the scenario described in this paper) the simulation can be used to decide what proportion of individuals to send to each the model.Note that the output of the function shows the value of p 1 , the proportion of class 2 individuals to be sent to queueing model 1.

A3. Reference
The primary tool that was used in the construction of the discrete event simulation model was the python library ciw .See Ciw's documentation for a more detailed explanation of how it works and what are its capabilities ( The Ciw library developers, 2020 ).
Find below a detailed list of the functions that were created for the simulation model: • build_model : Builds a ciw object that represents a model of a queueing network with two waiting spaces.
• build_custom_node : Build a custom node to replace the default ciw.Node.Inherits from the original ciw.Node class and replaces two methods.
• simulate_model : Simulate the model by using the custom node and returning the simulation object Obtains the optimal distribution of class 2 individuals such that the blocking times in the two systems are identical and thus minimised

A4. Explanation
Based on Ciw's functionality the simulation model stores all data records in a Record object.For every event that takes place a record is created with all the relevant information.For this specific library, the records that are stored, along with the range of values that they can take are as follows: • id_number ∈ R .

Mean waiting time
The recursive formula described here is the origin of the closedform formula described in Section 3.1 .
To calculate the mean waiting time one must first identify the set of states (u, v ) where a wait will occur.For this particular Markov chain, this points to all states that satisfy v > C i.e. all states where the number of individuals in the service centre exceed the number of servers.The set of such states is defined as waiting states and can be denoted as a subset of all the states, where: Additionally, there are certain states in the model where arrivals cannot occur.A type 1 individual cannot arrive whenever the model is at any state (u, N) for all u where N is the system capacity.Equivalently, a type 2 individual cannot arrive in the model when the model is at any state (M, v ) for all v ≥ T .Therefore the set of all such states that an arrival may occur are defined as accepting states and are denoted as: Moreover, another element that needs to be considered is the expected waiting time In order to do so a variation of the Markov model has to be considered where when the individual arrives at any of the states of the model no more arrivals can occur after that.
Thus, one may acquire the desired time by calculating the inverse of the sum of the out-flow rate of that state.Since arrivals are ignored though the only way to exit the state will only be via The equivalent matrix notation of the linear system of Eqs. ( 44) -( 52) is given by Zx = y , where: The elements of the matrix Z can be acquired using Z i j defined in Eq. ( 17) where i and j are states (u i , v i ) , (u j , v j ) ∈ S b (14) .
Thus, having calculated the mean blocking time for all blocking states b(u, v ) , they can be combined together in a formula.Using the arriving states A 2 defined in Section 3.2 by Eq. 12 the resultant formula for the mean blocking time is given by:

revisited)
To illustrate how the described formula works consider a Markov model where C = 2 , T = 2 , N = 4 , M = 2 ( Fig. 14 ).The equations that correspond to such a model are shown in ( 53) -( 58) and their equivalent matrix notation form is shown in (59) .

Appendix D. Mean proportion of individuals within target
In order to consider such measure though one would need to obtain the distribution of time in the system for all individuals.The complexity of such task is caused by the fact that different individuals arrive at different states of the Markov model.Consider the case when an arrival occurs when the model is at a specific state.
Time distribution at specific state (1 server): Consider the Markov model of Fig. 15 with one server and a threshold of two individuals.Assume that an individual of the first type arrives when the model is at state (0 , 3) , thus forcing the model to move to state (0 , 4) .The distribution of the time needed for the specified individual to exit the system from state (0 , 4) is given by the sum of exponentially distributed random variables with the same parameter μ.The sum of such random variables forms an Erlang distribution which is defined by the number of random variables that are added and their exponential parameter.Note here that these random variables represent the individual's pathway from the perspective of the individual.Thus, X i represents the time that it takes to move from the i th position of the queue to the (i − 1) th position (i.e. for someone in front of them to finish their service) and X 0 is the time it takes to move from having a service to exiting the system.
Thus, the waiting and service time of an individual in the model of Fig. 15 can be captured by an erlang distributed random variable.The general CDF of the erlang distribution Erlang(k, μ) is given by: Unfortunately, the erlang distribution can only be used for the sum of identically distributed random variables from the exponential distribution.Therefore, this approach cannot be used when one of the random variables has a different parameter than the others.In fact the only case where it can be used is only when the number of servers are C = 1 , or when an individual arrives and goes straight to service (i.e. when there is no other individual waiting and there is an empty server).
Time distribution at a state (multiple servers): Fig. 16 represents the same Markov model as Fig. 15 with the only exception that there are 2 servers here.By applying the same logic, assuming that an individual arrives at state (0 , 4) , the sum of the following random variables arises.
Since these exponentially distributed random variables do not share the same parameter, an erlang distribution cannot be used.In fact, the problem can now be viewed either as the sum of exponentially distributed random variables with different parameters   or as the sum of erlang distributed random variables.The sum of erlang distributed random variables is said to follow the hypoexponential distribution.The hypoexponential distribution is defined with two vectors of size equal to the number of Erlang random variables ( Akkouchi, 2008, Khaled, Kadri & Kadry, 2013 ).The vector r contains all the k -values of the erlang distributions and λ is a vector of the distinct parameters as illustrated in Eq. (63) .
Equivalently, for this particular example: = H ∼ Hypo((2 , 1) , (2 μ, μ)) Therefore, the CDF of this distribution can be used to get the probability of the time in spent in the system being less than a given target.The general CDF of the hypoexponential distribution Hypo( r , λ) , is given by the following expression ( Favaro & Walker, 2010 ): The computation of the derivative makes equation (65) computationally expensive.In Legros and Jouini (2015) an alternative linear version of that CDF is explored via matrix analysis, and is given by the following formula: Specific CDF of hypoexponential distribution: Eqs. ( 65) and (66) refers to the general CDF of the hypoexponential distribution where the size of the vector parameters can be of any size ( Favaro & Walker, 2010 ).In the Markov chain models described in Figs. 15 and 16 the parameter vectors of the hypoexponential distribution are of size two, and in fact, for any possible version of the investigated Markov chain model the vectors can only be of size two.This is true since for any dimensions of this Markov chain model there will always be at most two distinct exponential parameters; the parameter for finishing a service ( μ) and the parameter for moving forward in the queue ( Cμ).For the case of C = 1 the hypoexponential distribution will not be used as this is equivalent to an erlang distribution.Therefore, by fixing the sizes of r and λ to 2, the following specific expression for the CDF of the hypoexponential distribution arises, where the derivative is removed: Note here that the only difference between Eqs. ( 65) and ( 67) is the function.The next part proves that the expression for can be simplified for the cases of k = 1 , 2 .Eq. ( 68) shows the expression to be proved.
Proof of Eq. ( 68) : This section aims to show that there exists a simplified version of Eq. ( 65) that is specific to the proposed Markov model.Function is defined using the parameter t and the variables k and l.Given the Markov model, the range of values that k and l can take can be bounded.First, from the range of the double summation in Eq. ( 65) , it can be seen that k = 1 , 2 , . . ., | r | .Now, | r | represents the size of the parameter vectors that, for the Markov model, will always be 2.That is because, for all the exponentially distributed random variables that are added together to form the new distribution, there only two distinct parameters, thus forming two erlang distributions.Therefore: k = 1 , 2 By observing Eq. ( 65) once more, the range of values that l takes are l = 1 , 2 , . . ., r k , where r 1 is subject to the individual's position in the queue and r 2 = 1 .In essence, the hypoexponential distribution will be used with these bounds: Thus the left hand side of Eq. ( 68) needs only to be defined for these bounds.The specific hypoexponential distribution investigated here is of the form Hypo((r 1 , 1)(λ 1 , λ 2 )) .Note the initial conditions λ 0 = 0 , r 0 = 1 defined in Eq. ( 65) also hold here.Thus the proof is split into two parts, for k = 1 and k = 2 .

Fig. 1 .
Fig. 1.A diagrammatic representation of the queueing model.The threshold T only applies to type 2 individuals.If the number of individuals in the hospital is T , only individuals of type 1 are accepted (at a rate λ 1 ) and individuals of type 2 (arriving at a rate λ 2 ) are blocked in the parking space.

Fig. 2 .
Fig. 2. General case of the Markov chain model.

Fig. 3 .
Fig. 3. Comparison of mean waiting time between values obtained from the Markov chain formula, values obtained from the truncated simulation and values obtained from the untruncated simulation.

Fig. 4 .
Fig. 4. Comparison of mean blocking time between values obtained from the Markov chain formula, values obtained from the truncated simulation and values obtained from the untruncated simulation.

Fig. 5 .
Fig. 5. Comparison of mean proportion of individuals within target time between values obtained from the Markov chain formula, values obtained from the truncated simulation and values obtained from the untruncated simulation.

Fig. 7 .
Fig. 7. Imperfect information Extensive Form Game between the distributor and the 2 queueing systems.

Fig. 10 .
Fig.10.The strategies played when running asymmetric replicator dynamics along with the compartmentalised price of anarchy of the blocking time at each iteration of the learning algorithm.After a number of iterations the arrival rate of ambulance patients is significantly increased to flood the system completely λ 2 = 24 .

Fig. 11 .
Fig.11.The strategies played when running asymmetric replicator dynamics along with the compartmentalised price of anarchy of the blocking time at each iteration of the learning algorithm.After a number of iterations the number of servers for both systems are increased by one.

Fig. 12 .
Fig.12.The strategies played when running asymmetric replicator dynamics along with the compartmentalised price of anarchy of the blocking time at each iteration of the learning algorithm.After a number of iterations the most dominant strategy is being penalised.

Fig. 13 .
Fig. 13.A diagrammatic representation of the queueing model example.

Fig. 17 .
Fig. 17.Comparison of mean waiting time for type 1 individuals between values obtained from the Markov chain formulas and values obtained from simulation.

Fig. 18 .
Fig. 18.Comparison of mean waiting time for type 2 individuals between values obtained from the Markov chain formulas and values obtained from simulation.

Fig. 19 .
Fig. 19.Comparison of proportion within target time for type 1 individuals between values obtained from the Markov chain formulas and values obtained from simulation.

Fig. 20 .
Fig. 20.Comparison of proportion within target time for type 2 individuals between values obtained from the Markov chain formulas and values obtained from simulation.

Table 1
Relative timings of the simulation and Markov chain model.
Fig.6.A diagrammatic representation of the game theoretic model.Patients arrive at the EMS at a rate of λ 2 and then a proportion of them are distributed to hospitals A ( p A ) and the remaining proportion to hospital B ( p B ) so that p A + p B = 1 .The corresponding arrival rates of type 2 patients to hospitals A and B are thus given by: p A λ 2 and p B λ 2 .