A hybrid simulation approach for planning health and social care services

ABSTRACT Discrete-event simulation (DES) has been recognised for many years as a powerful tool to support the commissioning and resourcing of health and social care services, due to its ability to capture real-world variability. However, the complex interactions between two distinct but clearly related processes, disease progression and care provision, can lead to such models being cumbersome and lacking in transparency. Representing disease progression as a series of queues and activities is not always intuitive to a non-modeller. This paper presents a novel hybrid simulation approach in which health status is modelled using statecharts, thus combining DES with agent-based simulation. This hybrid approach allows disease progression to be modelled in a more natural way, keeping the overall model structure relatively simple. The approach is illustrated by a case study that evaluates the impact of telecare services for supporting people with dementia.


Introduction
Discrete-event simulation (DES) is a stochastic, individual-level approach that has been widely used for decades to model healthcare delivery systems from an operational and tactical perspective. DES depicts such systems as networks of queues and activities through which patients flow, and hence is ideally suited for modelling service redesign, capacity planning and resource allocation problems. However, the services an individual patient requires, the pathway they take through the healthcare system, and how long they spend in a particular part of the system, will depend on that individual's health status (and vice versa). Health status can be represented in DES as an attribute of the patient entity in exactly the same way as age, gender and other individual characteristics, but normally attributes can only be observed or modified at the start or end of an activity. In reality, disease progression is a completely separate process that operates in parallel to patient flow and can influence it. It can of course be modelled directly in DES, but treating health status as a series of queues and activities is not always intuitive to stakeholders. Moreover, the complex interactions between the processes of patient flow and disease progression can lead to models being cumbersome, especially when a change in health status occurs during a patient flow activity and leads to that activity being interrupted: for example, a postoperative patient who develops a wound infection on the ward during the activity 'post-op stay' may have to return to theatre for remedial surgery.
This paper presents a novel simulation approach in which health status is modelled using statecharts, i.e., combining DES with agent-based simulation. This hybrid approach allows disease progression to be modelled in a more natural way, keeping the overall model structure relatively simple and avoiding the need for additional code to model interrupted activities. The approach is implemented in the DES software Simul8 and is illustrated by a case study that evaluates the impact of telecare services for supporting people with dementia.
The remainder of the paper is structured as follows. Section 2 contains a very brief introduction to agent-based and hybrid simulation. Section 3 explains the context of the case study and reviews the literature on simulation in social care. Section 4 provides some background information about telecare and its use in dementia, and section 5 presents the TeleDem model, describing in detail how the hybrid approach was implemented. The results from a set of experiments that evaluate the impact of different levels of telecare provision on demand for residential care are presented in section 6. Section 7 discusses the implications of these findings for policy makers, and section 8 concludes the paper with some general reflections on the modelling approach and its wider applicability beyond healthcare.

Agent-based and hybrid simulation
An agent-based simulation (ABS) model consists of a set of agents and an environment in which they exist. Agents are assigned individual attributes and rules that determine their behaviour. Agents are able to learn, and adapt their behaviour, from interactions with their environment and with other agents. Unlike DES, which takes a process view, ABS models a system from the 'ground up' -"agent by agent and interaction by interaction" (Macal & North, 2010). However, in their pure form ABS models require a large amount of data and/or assumptions about the rules that govern behaviour which can limit their realworld applicability (Sally Brailsford, 2014). A feature of ABS is the statechart, a graphical representation of the various states an agent may inhabit together with transition rules for how an agent exits one state and enters another. Statecharts have proved an effective tool for modelling disease progression: their compact yet expressive nature means that they are able to capture complex behaviour with relatively few elements (Vickland & Brodaty, 2008).
The use of hybrid simulation, defined as a modelling approach that combines two or more of DES, ABS, and system dynamics (SD), has experienced massive growth in recent years. According to a review paper (S. C. Brailsford et al., 2019) this is due in part to the increasing popularity of AnyLogic, currently the only commercial simulation software that allows the modeller to switch between all three methods in the same environment: 47 of the 139 models reviewed used AnyLogic and only 13 used the second most popular software tool, the DES software Arena. Healthcare was identified as the most popular application area, with 31 papers (the second most popular was supply chain management, with 26), possibly because healthcare problems typically ' . . . have multiple aspects, and it is rarely possible to capture all of them in one single model using only one method.' (S. C. Brailsford et al., 2019, p. 728). Arguably, complex supply chains share the same characteristics. Of course, the concept of combining modelling methods is far from new. Jackson and Keys (1984) argue that since all OR methods have different strengths and weaknesses, mixing methods offers the potential to overcome some of the drawbacks of using a single approach.
S. C. Brailsford et al. (2019) discuss the technical challenges of implementing hybrid simulation in commercial software tools that were originally designed for a single method. Such tools are constantly evolving as vendors try to respond to demand: most SD packages can now employ probabilistic sampling, and some DES tools can incorporate continuous flows. Following a lively debate about the future of DES at the 2010 UK OR Society Simulation Workshop, Siebers et al. (2010) suggested that DES software could be improved by including statecharts. It seems the vendors of Simul8 were listening: the developers subsequently incorporated statechart functionality into the Professional version of the package. This is an example of the "enrichment" category of hybridisation as defined by Morgan et al. (2017), where one dominant method is used with limited use of other methods(s). It is within this Simul8 environment that the TeleDem model presented in this paper was developed.

Simulation in social care
Although simulation has been widely used in healthcare for many decades (Katsaliaki & Mustafee, 2011), there are relatively few applications in community and social care (Onggo, 2012). Like healthcare, social care systems are complex, involving multiple stakeholders, often with conflicting objectives (Onggo, 2012). However social care systems interact with many other complex systems, such as local government services and the healthcare system; funding is typically more complicated, and in the UK care is often delivered by a variety of providers, including charities, friends and family as well as public and private organisations.
The most commonly used simulation approaches in the area of social care are system dynamics (SD) and DES, although microsimulation has also been used (Jagger et al., 2009). SD has been used to examine policies focused on admission prevention and delayed discharge (Wolstenholme et al., 2004), telecare (Bayer et al., 2007), and more widely older people's care (Ansah et al., 2013;Desai et al., 2008;Thompson et al., 2013). Examples of DES applied to social care include studies on hospital-at-home services (Lebcir et al., 2017), improving efficiency in service delivery (Harpring et al., 2014), and analysing admission policies and waiting list management strategies in home care services (Maroufkhani et al., 2016). Larrañaga et al. (2018) used DES to explore the sustainability of an integrated health and social care programme for heart failure, while Patel et al. (2020) used DES to model the societal costs of stroke in the UK, taking into account hospital care, community care, institutional care and informal care costs.
Only one of the 31 healthcare-related papers in S. C. Brailsford et al.'s (2019) review of hybrid simulation concerned social care. This was Viana et al., 2012), which describes a hybrid model for health and social care provision for the eye condition age-related macular degeneration. Since this review, Standfield et al. (2019) have published a further example of hybrid social care modelling, where microsimulation was used in combination with DES to estimate the future costs associated with dementia care in Australia.

Dementia and telecare
Dementias, such as Alzheimer's disease or vascular dementia, are a major cause of disability and dependence amongst older people (Singh et al., 2014). Research indicates that 5% of people over the age of 65 years have some form of dementia, with the prevalence increasing significantly with age, to one in three by 90 years (Iliffe & Drennan, 2001). While all dementias are chronic and progressive, resulting in deterioration in cognition, behaviour, and daily functioning (Bjørneby et al., 1999), the rate of progression varies greatly between individuals (Barocco et al., 2017). As their dementia advances a person will become increasingly dependent on other people for support and supervision in order to manage day to day activities . As there is currently no effective long-term medical treatment, dementia is predominantly a social care rather than a healthcare issue up until the very late stages and informal care is often provided by friends and family (Robinson et al., 2015). The emotional and physical stress that informal carers can experience often increases over time as the illness progresses. Carer "burden" has consistently been shown to be one of the main predictors of institutional care admission for people with dementia (Charlesworth, 2010;Hébert et al., 2001;Zarit et al., 1986). Stage of disease progression, level of dependency, and carer capacity to cope are core influencers over demand for social care services in order to support people to remain living at home. The complexity of the dementia pathway exemplifies a planning challenge encountered in other areas of health and social care. Variability in disease progression, combined with the mutual interaction between care processes and health status makes anticipating the impact of new services particularly difficult to predict.
In recent years policy makers and care professionals have been looking for new ways to provide care to frail elderly people including those with dementia, offering greater choice over care pathways, while containing costs. One of these options is telecare, which makes use of personal alarms and sensors such as fall detectors or environmental detectors to help people live independently in the early stages of dementia. Telecare works by monitoring for changes and warning the people themselves or raising an alert at a control centre. It aims to help people with dementia to maintain their independence, delaying or even eliminating the need for institutional care (Siotia & Simpson, 2008). Evidence indicates that telecare can reduce the potential for accidents; mediate risks in the home (Clark, 2009); provide service users with increased choice for condition management (Knipscheer, 1994); maintain health through the early detection of deterioration, facilitating accelerated diagnosis and treatment (Wright, 1998); and alleviate carer stress (Alaszewski & Cappello, 2006). There are social and economic (Dowd et al., 2018) and quality of life (Jing et al. (2016) advantages to avoiding institutional care. Studies have associated institutional placement with both accelerated cognitive decline (Wilson et al., 2007), and reduced life expectancy (McClendon et al., 2006). A preference for care at home therefore underpins community care policy across Europe (Tucker et al., 2008;Wübker et al., 2015) and the promotion of telecare as a way to support people in their own home.
Despite this extensive body of literature describing the benefits of telecare, much of this is based upon anecdotes and short-term trials, which has been insufficient to encourage widespread implementation. Simulation modelling provides an opportunity for policy makers to explore the impact of mainstream telecare delivery and build an evidence base for informed decision making.

The teledem model
The aim of the TeleDem model is to evaluate the impact of telecare on the numbers of people with dementia able to remain living within their own homes, rather than entering institutional care. It was developed in consultation with domain experts involved in the development and delivery of telecare; clearly, it represents a simplified version of the care system. DES is used to model the dementia care pathway, i.e., the services that individuals receive, while statecharts are used to model the individual's dementia state and level of dependency, and the level of stress experienced by their carer. Figure 1 presents the conceptual DES model showing the dementia care pathway, while Figure 2 shows the attributes of each individual, including the dementia statechart (in green) and the carer burden statechart (in blue and purple).These services do not relate specifically to any region or local authority, but are representative of social care provision in the UK. People enter the model at the point that they receive their initial diagnosis. At this point, they can be in any of three dementia states: mild, moderate or severe. On "arrival", each person is assigned a set of unique characteristics, sampled from the UK population aged over 65 with dementia (Prince et al., 2014), and are assigned to the relevant state within the dementia statechart, which drives disease progression. Changes in dementia state directly influence the level of dependency, which in turn controls the carer burden statechart.
An individual's unique characteristics govern how they are routed around the DES model. If they do not require formal support from a care service at the time of diagnosis, they return home. If they do require assistance, the level of care they require is reviewed and they are referred to one of the six care services shown in Table 1. The first four are provided in the person's own home, whereas the last two are provided in an institutional care setting. In practice, the number of visits per day and the level of support provided under Community Care varies enormously from person to person, and this is sampled in the model for each individual -see Appendix 1 for details of the probability distributions used.
A person's progression through different states in a statechart is independent of their position within the DES, although movement around the DES can be triggered by changes in state. For instance, if a person dies, they are immediately removed from the DES; if a person progresses to a more severe stage of disease, their care needs will change. If their informal carer can no longer cope, the patient will require referral to a new service and possibly even a move into institutional care, even if their dementia state and their care needs are unchanged. This allows the model to reflect the reality of the care pathway, where the outcomes of assessments, reviews, and referrals to care services are based on each person's current situation.  The statecharts update on a daily basis as the DES model runs. As an individual transitions through the stages of dementia, their level of dependency increases. The model distinguishes six levels of dependency, i.e., two levels of dependency within each of the three dementia stages. The number of hours of support the person requires and the level of carer burden experienced are determined by their level of dependency. Carer burden, measured as a Zarit Burden Inventory (ZBI) score (Zarit et al., 1986), and care status directly influence each person's position within the Carer Burden statechart. As carer burden increases, so does the probability of carer collapse and institutional care placement (Hébert et al., 2001).
When a state change occurs the person is routed to the care service that best meets their needs. ZBI scores within the model are recalculated on referral to telecare-based services, to reflect the influence of telecare over reducing carer burden. This effect only remains while the person is in receipt of a telecare-based care service.

Model boundaries and input data
The TeleDem model does not represent a specific locality or population. Characteristics of the cohort were assigned to represent people with Alzheimer's aged 65 years and older from a city with a population of approximately 250,000 people. The model focuses on the social care system and does not include any interactions with the healthcare system: neither does it examine service capacities or waiting times. Therefore, the availability of all care services (apart from telecare) are assumed to be unconstrained, allowing the model to focus solely on the impact of telecare on the demand for institutional care. Daily occupancy of residential and nursing home beds is the key performance indicator.
The model parameters were mainly derived from the literature, although where data could not be found domain experts provided estimates. Appendix 1 details the data sources by parameter, and Appendix 2 contains a detailed list of all the model assumptions. Figure 3 shows a screenshot of the TeleDem model, with the statecharts to the right of the DES. During a run, each coloured rectangle in the statechart displays the number of patients currently in that state: since this screenshot shows the state of the model at time zero (i.e., before any patients enter the system) they are all zero. The six services are represented as activities, each of which requires a resource (basic telecare equipment, complex telecare equipment, community care service, residential care space and nursing home beds). The simulation uses days as its time unit.

Model implementation
Technically, states are implemented in Simul8 as "work item labels", i.e., attributes of entities. These labels are given a value of zero or one depending on whether the entity is in that state or not. The modeller specifies the rules governing state transitions, and also how frequently the rules are checked. If a rule is satisfied, the corresponding state change takes place. In the case of TeleDem, checks were performed on a daily basis.
Unlike standard Simul8, where (for example) routing decisions can only be made after an activity has ended, statecharts allow routing decisions to be made at any point, based on the entity's status. Events can be triggered based on a person's state -so using our earlier example of a postoperative patient -if the patient develops a wound infection in the middle of the activity "Post-op stay", they can immediately be sent to a queue for remedial surgery, rather than having to wait until the end of their previously sampled activity duration. Statecharts provide a simple and elegant mechanism for implementing interrupted activities. The model is available for download as online appendix.

Verification and validation
The model was built incrementally, which allowed each part to be run independently and tested so that problems could be identified and resolved quickly. The structure and logic of the model were checked with experts in Gerontology and dementia care, as were the simulated patterns of referral. Experiments were conducted to validate model behaviour against the published literature. For the purposes of validation, the model was run for 25 years with a fixed arrival rate of one person per day and all arrivals in state Mild, in order to model a cohort from dementia onset to death and compare with the literature. The TeleDem survival times following a diagnosis of dementia were compared to those published in  Xie et al. (2008). Table 2 shows median survival by age band produced by the TeleDem Simulation, and in Xie et al. (2008). At the end of the run the TeleDem Simulation produced similar proportions of the population at each stage of dementia as reported in the literature (Prince et al., 2014), as shown in Table 3.
At the end of the 25-year run 64% of the modelled population were living in the community and 36% were living in institutional settings. This is consistent with the estimates produced by Prince et al. (2014), who suggested that in 2014, out of the 805,369 people with dementia in the UK, 61% were living in the community, while 39% lived in institutional care (either residential care homes or nursing homes). The model results were also used to compare where people were living at each stage of dementia, as shown in Table 4.

Experimentation
In principle, the TeleDem Simulation can be used to test an extensive range of policy options and scenarios by varying the model inputs and parameters. In this paper we explore the impact on resource use of varying the availability of basic versus complex telecare. Due to a lack of consensus on the proportion of people with dementia who would be suitable for and accept telecare, two scenarios are considered, under which 90% (optimistic) or 10% (conservative) of all referrals to community care will result in the uptake of telecare. For both optimistic and conservative scenarios, we tested three policy options for telecare services to which people with dementia can be referred: basic telecare only (option 1), complex telecare only (option 2), and both basic and complex telecare (option 3). Therefore, a total of six simulation experiments were conducted and tested against a baseline scenario in which neither basic nor complex telecare services were available. In all the experiments, the simulation duration was 40 years which included a warm-up period of 10 years to allow the model to reach steady state. Given that disease progression is measured in years and some activities have equally long durations, a run length of 30 years was considered appropriate. Each experiment used a trial of 35 runs, calculated using Simul8ʹs built-in runs calculator.
In the baseline scenario, the mean number of days spent in institutional care for those entering institutional care was 1,174 days (3.2 years), with an average stay of 1,027 days (2.8 years) in residential care and 551 days (1.5 years) in nursing homes. The daily resource utilisation is summarised in Table 5.

Results
The results from these six scenarios and the baseline are summarised in Table 6. The average number of basic telecare resources in use per day under policy options 1 and 3 are very similar, indicating that the uptake of basic   telecare is relatively unaffected by the addition of complex telecare. This is consistent across both conservative and optimistic scenarios. Similarly, in the conservative scenario, the average number of complex telecare resources in use per day appears relatively constant, whether offered in isolation or in combination with basic telecare. However, in the optimistic scenario the uptake of complex telecare is higher under policy option 3 compared with policy option 2. A possible explanation for this is that basic telecare reduces the number of people entering institutional care in the early stages of dementia, and thus a larger pool of people remain in the community and are referred to complex telecare services as their disease progresses. Policy options that include basic telecare reduce the average daily utilisation of community care packages, compared with the baseline. This is because basic telecare offers a direct alternative to community care in the earlier stages of dementia. In contrast, complex telecare increases the uptake of community care, because (when combined with community care) it reduces the number of admissions to residential care.
All telecare policy options reduce the average daily occupancy of residential care home places and nursing home beds. Unsurprisingly, the impact of telecare is greater under the optimistic scenario. Policy option 3 has the greatest impact. However, option 1 has a greater impact on the average occupancy of residential care places than option 2. The converse is true for nursing home beds, with Complex Only having the greater impact. Under the optimistic scenario, policy option 3 has the greatest impact on demand for institutional care in general, with a reduction in the average daily occupancy of 10.16%. Policy option 2 had the second largest impact, with a reduction of 5.69% (see, Figure 4).

Cost implications
In both the conservative and optimistic scenarios, telecare reduces the daily use of institutional care resources. Figure 5 presents the overall costs of delivering telecare, irrespective of whether the local authority pays for care, or the person covers the cost themselves.
The results show that policy option 3 leads to the greatest savings. However, the savings associated with option 2 are close behind. Offering both complex and basic telecare together reduces annual expenditure on care provision for the dementia care pathway by 1.64% in the conservative scenario, compared to baseline. This equates to an annual saving of nearly £2.5 m for this population. The equivalent reduction of 12.21% in the optimistic scenario represents an annual saving of around £18.5 m. The introduction of basic telecare results in annual savings just over £2 m (1.38% reduction compared to baseline) under the conservative scenario and around £16.5 m (10.93% reduction compared to baseline) under the optimistic scenario. By comparison, complex telecare alone has the smallest impact on spending, with a reduction of 0.38% (roughly £2 m) in the conservative scenario and 1.47% (£2.2 m) under the optimistic scenario, when compared to baseline. The additional cost of supporting a larger proportion of the dementia population with complex telecare equipment combined with community care limits the impact of any savings resulting from lower occupancy of institutional care beds.

Implications for dementia care
The scenarios tested in section 6 are of course illustrative, but the results show that telecare could provide significant cost savings even if uptake is relatively low, and that for local authorities to maximise the impact of telecare services, they need to invest in both basic and complex equipment. If, however, a decision maker was only able to choose one of these, they should prioritise basic equipment offered in the earlier stages  of dementia, therefore preventing unnecessary early admissions, as the cost savings under policy option 1 are greater than those under option 2.
The TeleDem simulation provides a platform to test different scenarios and policy options, which would enable decision makers to understand the impact of changes on the wider system and the cost implications. It allows the exploration of policy options over extended time frames, which provides the opportunity to investigate the longer-term implications of the interventions, which can be missed during short term pilot studies.

Implications for modelling: Combining DES and statecharts
The TeleDem model demonstrates how simulation can take into account both individual level variability in terms of disease progression (and, in the case of dementia, carer burden) as well as process variability along the care pathway, while retaining a considerable degree of transparency for non-technical stakeholders. The progression of disease in an individual determines what care they require, but the rate at which diseases progress can vary greatly between individuals. Building disease progression into a standard DES model requires complex logic, particularly when treatment process activities have to be interrupted if a patient's health status changes. This is less of an issue if the process activities are all relatively short in duration, as for example, in a model of an Emergency Department: in such cases it is a reasonable approximation to reality to wait until the end of an activity to check the patient's health status, and then route them to the appropriate place in the care pathway. However, if process activities are measured in months or years rather than minutes, waiting until such activities end before reassessing the patient's care needs is not a realistic way to represent what actually happens in practice. Hence statecharts lend themselves particularly well to long-term conditions like dementia, where patients (or, in the case of social care, service users) follow a care pathway for a number of years and receive the same service for an extended period if their health remains the same.
While healthcare is perhaps the most obvious application area where the route that an entity follows through some queueing network depends on its status, and status can change at any time independently of the entity's location within the queueing network, the same is true in other application domains. A vehicle transporting perishable goods in a distribution network may break down en route if it has not been properly maintained, or it may be involved in a road traffic accident: in either case the goods may degrade and lose value (partially or completely) due to the delay. Items in a manufacturing process may change priority if an order is received:activities already under  way on low-priority items may need to be interrupted in order to process high-priority items, or different machines used for certain activities to expedite production. Statecharts that represent the current shelf life of perishable goods, or the current priority status of items, provide a realistic way to model such systems. Combining statecharts with DES allows individual level realism to be combined with the structural robustness of traditional DES, but, crucially, in a format that is transparent and easy to communicate to stakeholders. (Continued).

Parameter
Description Source Transition to "High Level of Unmet Need" Calculation relating to unmet need. This is calculated by subtracting the number of hours of care an informal carer can provide from the number of hours of care the person requires. If the resulting value is greater than 0, the person has unmet need and therefore needs to be referred for additional support from a care service. The person will then transition back into a "Coping state" within the Carer Burden Statechart once the unmet need has been addressed.
Not Applicable.
Transition to "Informal Carer Seeking Extra Support" Transitioning to "Seeking Extra Support" relates to people with informal carers who return home without a community care service following diagnosis, or people who are referred to "basic telecare only". These people are assigned a transition rate that increases in line with their increasing dependency. Therefore, the higher their level of dependency, the more likely the person is to transition to "seeking extra support". Once an appropriate referral has been made to address the unmet need, or requirement for extra support, then the level of burden experienced by the person's carer is reset to one of the core "Coping" carer nodes. Scherer et al. (2008).
Transition to "Informal Care Collapse" Increased likelihood of carer collapse based on increasing levels of carer burden as dependency levels increase.

Routing following diagnosis
The probability of someone requiring support from a formal care service following diagnosis is assigned based on estimates derived from the results of the DEMHOM questionnaire . .
Referrals to community care vs institutional care