Evaluation of appointment scheduling rules: A multi-performance measurement approach☆
Introduction
Professionals in health care and other services face the problem of allocating time windows to customers. This allocation can be done by means of Appointment Scheduling Rules (ASRs). ASRs determine when a customer is to receive service during a service session. Although the literature on ASRs is mainly focused on health care (e.g., in an emergency department, a doctor’s office, or an operating theater), the research topic is generic and applicable in many industries: attorneys, faculty receiving students, tax accountants, consultants, barbers, service centers, trailers at receiving bays, and many others.
With increasing customer expectations on being served quickly, both in health care and in other service industries, timeliness of appointments is crucial [31], [33]. Moreover, timely delivery of care has been shown to reduce mortality and morbidity associated with a variety of medical conditions [69]. The customer waiting time consequently is a relevant performance measure. A second important objective of appointment scheduling has to do with the efficiency of the service. For private companies, the impetus to efficiency comes naturally. However, health care systems are under pressure to use their capacity effectively and efficiently [34]. Doctors’ (or more general servers’) idle time and overtime are hereby important performance measures. In addition, a distinction needs to be made between private healthcare institutions (who may focus more on patient waiting time) and public care systems (where taxpayer money needs to be spent as efficiently as possible).
The literature on appointment system design can broadly be divided into two classes. In the first class the objective is to give individual patients a fixed appointment time, that means, we have to decide on the order the appointments will be scheduled. It comes down to a sequencing decision. Patient class information will be used for sequencing purposes. In the second class of studies, authors try to find the best appointment rule. Here we have to decide on the length of the appointment intervals (fixed or variable) and the block sizes (number of patients scheduled to each appointment slot, individual or multiple). In this case, patient information such as no-shows, patient unpunctuality and other disrupting factors are taken care of by adjusting the appointment intervals and/or the block sizes. In this article, we focus on the second class, namely on ASRs but in a complex environment characterized by unpunctuality, no-shows, and server lateness. In literature there is a debate on the question which approach is best, sequencing or appointment rules. We opt for appointment rules because we believe that ASRs will perform more robustly in complex environments. Cayirli et al. [11], who combine both approaches, clearly mention that sequencing-based appointment systems are less flexible than those that assign patients on a first-call, first-appointment basis. They continue the argument by stating that the biggest challenge for future research will be to find new appointment systems that will perform more robustly across different clinical environments and patient panels. We position our paper in the ASR literature, and focus on identifying good ASRs that have robust performance in a variety of settings. Of course, we fully appreciate the work done by authors focusing on sequencing. We refer to the excellent paper of Deceuninck et al. [22]. Their approach allows to take prior individual knowledge about the patients into account. If such information is available and correctly exploited, the sequencing approach may lead to substantial cost reductions. A similar conclusion was obtained by Cayirli et al. [10], [11] claiming that the use of patient classification (e.g., distinguishing between new and return patients, or taking into account general patient and clinician characteristics) improves performance. Kuiper et al. [44] also deal with the issue of sequencing (they call it dealing with distinguishing varieties). They conclude, however, that process simplification is of greater importance. It is clear that there is a lively discussion in the literature on the use of appointment scheduling rules on the one hand, and the sequencing-based appointment systems on the other hand.
The objective of this article is to identify ASRs that simultaneously minimize customer waiting time, server idle time, and overtime. This has to be done in an environment where both demand and supply characteristics are highly uncertain, and subject to many sources of variability. For this purpose, we develop an analytical model to determine the best appointment policy under a wide range of assumptions. In contrast to most of the literature (e.g., [37], [46]), we do not rely on simulation but use a Discrete-Time Markov Chain (DTMC) to model the Appointment System (AS).
ASRs determine the planned (scheduled) arrival rate of customers during a service session. The actual arrival time may differ from the planned arrival time. Therefore, we assign each customer a probability of being too late or too early. In addition, we assign each customer a probability of not showing up. Because of the no-show problem (i.e., customers not showing up for their appointment), the actual number of customer arrivals is unknown, even if the number of customers per session is fixed and predetermined. The performance of ASRs is not only influenced by the arrival rate and service rate characteristics. Other types of outages during the service session are also important. We therefore allow for delays at the start of a service session due to late arrival of a server, or due to setup activities at the start of a session. We also allow preemptive and non-preemptive interruptions during the service session (e.g., it is well known that scheduled appointments can be disrupted by emergencies). All these extensions allow us to model real-life appointment systems, and to identify ASRs that have a robust performance across different settings. It is interesting to mention at this point that appointment scheduling approaches may have limitations in real healthcare applications. Issues not included, such as the availability of resources, the exploitation of flexibility of resources, the utilization level of resources, may however improve the performance of appointment scheduling. We refer to Kuiper et al. [44] for an in-depth discussion of this issue and the suggestions made for immediate improvement in practice. Similar concerns can be found in Mondschein and Weintraub [57].
We develop an analytical model that uses an efficient (in terms of computational and memory requirements) algorithm to assess the performance of ASRs. The validity and accuracy of the model are supported by a simulation study. We use the model to assess the performance of a set of 314 ASRs in an elaborate computational experiment. To compare the performance of these ASRs (in terms of waiting time, idle time, and overtime), we apply Data Envelopment Analysis (DEA).
The contribution of this article is threefold:
- 1.
We develop a new analytical model to assess the performance of an ASR in a general setting.
- 2.
We perform an elaborate computational experiment to analyze the performance of a large number of ASRs. As such, we provide insight in what ASR is best in a given environment. This analysis is particularly useful for smaller general practices and hospitals that do not have the time/resources to optimize their appointment system. We also confirm, and unify, the findings of several other studies in the field of appointment scheduling.
- 3.
We use DEA to identify the best ASR based on multiple performance measures. The use of DEA in appointment scheduling is novel and allows to overcome limitations of traditional approaches that require to fix subjective weights for different objectives (i.e., waiting time, idle time, and overtime). As a non-parametric method, DEA does not require to specify weights (i.e., it allows for a fair comparison of ASRs). DEA also offers tools to measure the robustness of the decision rules, we refer to the Maverick score that is used in this paper.
We also provide a number of important managerial insights. The first insight is that simple individual ASRs, like the Bailey–Welch rule, perform very well, certainly in the case where only a small number of customers needs to be scheduled. Secondly, we find that Variable Interval (VI) (or dome-shaped) ASRs are among the best performing ASRs and that their performance is robust over complex and dynamic environments. This means these VI ASRs are recommended in AS where the environmental variables are prone to change. The third important insight is that the relative performance of ASRs drastically changes when waiting time becomes more important. Thus, if customer waiting time is a crucial factor for success (and more relevant than overtime or idle time), managers may want to consider an ASR that results in smaller waiting times, even if this results in more idle time and/or overtime. Finally, our results suggest that customer punctuality, although having a significant impact, is less important than other environmental variables such as no-shows and service variability for the overall performance of ASRs.
This article is organized as follows. Section 2 provides a description of the problem setting. The literature on appointment systems is discussed in Section 3. Section 4 defines the basic processes that govern the appointment system, and Section 5 presents the basic model. The design and the results of the computational experiment are discussed in Section 6. Section 7 concludes.
Section snippets
Problem description
ASRs are used to schedule the servicing of a given number of customers during a service session. Complexity is introduced in the form of so-called “environmental factors”. An extensive overview of such environmental factors is provided in Cayirli and Veral [9], Gupta and Denton [32], and Ahmadi-Javid et al. [1]. In this article, we take the following factors into account:
- •
Customers are allowed to arrive early or late (“customer unpunctuality”), or may even fail to show up.
- •
Each customer has a
Literature review
Appointment systems have been studied extensively over the past 50 years. They arise in many contexts. In transportation, AS have been used to schedule the arrival of cargo ships and trucks at ports [29], [58], [65], to schedule railway operations [45], [80], and to allocate airport slots [52], [53]. AS have also been adopted in telecommunication networks to schedule data transmissions [64], [74]. In manufacturing settings, AS have been used to schedule deliveries in just-in-time inventory
Definitions
In this section, we classify the different ASRs considered in our study. In addition, we define the basic processes that govern the AS, and introduce a discretization procedure that allows us to obtain the discrete distributions of customer service and arrival times. These discrete distributions are used in the DTMC that is used to model the AS.
Model
In this section we discuss the DTMC that is used to model the AS and that allows us to obtain the performance measures. To efficiently compute these performance measures we use an algorithm that is also introduced here.
Computational experiment
The model has been verified by means of an elaborate simulation study in which each of the operating environments is simulated using 5,000,000 simulation iterations [19]. Various values of Δ were tested, and a value of was shown to provide a sufficient level of accuracy while maintaining computational performance. As such, in the upcoming experiment we let . Note that the 5-min intervals have also been used by other researchers, such as Klassen and Yoogalingam [40]. It should be
Conclusion
Appointment scheduling rules are used to determine the point in time at which a customer is to receive service during a service session. ASRs are commonly applied in service and manufacturing industries (e.g., healthcare or after-sales service).
We develop an analytical model that uses a DTMC and an efficient algorithm to assess the performance (in terms of customer waiting time, server idle time, and server overtime) of ASRs in a wide variety of settings. More specifically, the model takes into
CRediT authorship contribution statement
Stefan Creemers: Conceptualization, Methodology, Software, Data curation, Writing - original draft. Marc R. Lambrecht: Supervision, Funding acquisition, Writing - review & editing. Jeroen Beliën: Methodology, Software, Writing - review & editing. Maud Van den Broeke: Formal analysis, Writing - review & editing.
References (82)
- et al.
Outpatient appointment systems in healthcare: a review of optimization studies
Eur J Oper Res
(2017) - et al.
Simulation modelling and analysis of appointment system performance for multiple classes of patients in a hospital: A case study
Oper Res Health Care
(2016) - et al.
Scheduling identical parallel machines to minimize total tardiness
Int J Prod Econ
(2008) - et al.
Sequencing surgical cases in a day-care environment: An exact branch-and-price approach
Comput Oper Res
(2009) - et al.
Choosing weights from alternative optimal solutions of dual multiplier models in DEA
Eur J Oper Res
(2007) Appointment-driven queueing systems
(2009)- et al.
Outpatient scheduling with unpunctual patients and no-shows
Eur J Oper Res
(2018) - et al.
A sequential bounding approach for optimal appointment scheduling
IIE Trans
(2003) - et al.
Efficient and cross-efficiency in DEA – derivations, meanings and uses
J Oper Res Soc
(1994) - et al.
A better hospital experience
(2007)
Appointment system design with interruptions and physician lateness
Int J Oper Prod Manag
Appointment scheduling in tandem-type service systems
Omega
Clinical appointment process, improvement through schedule defragmentation
IEEE Eng Med Biol Mag
Block appointment systems for outpatient clinics with multiple doctors
J Oper Res Soc
Queues with scheduled arrivals: A correction, simplification and extension
J R Stat Soc Ser B
Appointment policies in service operations: A critical analysis of the economic framework
Prod Oper Manag
Scheduling doctors’ appointments: Optimal and empirically-based heuristic policies
IIE Trans
Using client-variance information to improve dynamic appointment scheduling performance
J Oper Manag
The performance of a generalized Bailey-Welch rule for outpatient appointment scheduling under inpatient and emergency demand
Health Care Manag Sci
Selecting a suitable appointment system in an outpatient setting
Med Care
An optimal scheduling model for ready mixed concrete supply with overtime considerations
Autom Construct
Appointment scheduling with no-shows and overbooking
Prod Oper Manag
A hybrid prediction model for no-shows and cancellations of outpatient appointments
IIE Trans Healthc Syst Eng
Out-patient queues at the Ibn-Rochd health centre
J Oper Res Soc
A study of queues and appointment systems in hospital outpatient departments, with special reference to waiting-times
J R Stat Soc Ser B
Outpatient scheduling in health care: A review of literature
Prod Oper Manag
Designing appointment scheduling systems for ambulatory care services
Health Care Manag Sci
Assessment of patient classification in appointment system design
Prod Oper Manag
A universal appointment rule in the presence of no-shows and walk-ins
Prod Oper Manag
A universal appointment rule with patient classification for service times, no-shows, and walk-ins
Serv Sci
Sequential clinical scheduling with patient no-shows and general service time distributions
IIE Trans
Super-efficiency in data envelopment analysis
Creating composite indicators with DEA and robustness analysis: The case of the technology achievement index
J Oper Res Soc
Data envelopment analysis (DEA) – thirty years on
Eur J Oper Res
An advanced queueing model to analyze appointment-driven service systems
Comput Oper Res
Queueing models for appointment-driven systems
Ann Oper Res
Make-to-order policies for a stochastic lot-sizing problem using overtime
Int J Prod Econ
Dynamic appointment scheduling of a stochastic server with uncertain demand
INFORMS J Comput
Patients’ waiting time and doctors’ idle time in the outpatient setting
Health Serv Res
Determination of optimal variable-sized multiple-block appointment systems
Oper Res
Cited by (0)
- ☆
This manuscript was processed by Associate Editor W. van den Heuvel.