https://orcid.org/0000-0003-2544-767X
Figures
Abstract
Introduction
The aim of our study was to assess the initial impact of COVID-19 on total publicly-funded direct healthcare costs and health services use in two Canadian provinces, Ontario and British Columbia (BC).
Methods
This retrospective repeated cross-sectional study used population-based administrative datasets, linked within each province, from January 1, 2018 to December 27, 2020. Interrupted time series analysis was used to estimate changes in the level and trends of weekly resource use and costs, with March 16–22, 2020 as the first pandemic week. Also, in each week of 2020, we identified cases with their first positive SARS-CoV-2 test and estimated their healthcare costs until death or December 27, 2020.
Results
The resources with the largest level declines (95% confidence interval) in use in the first pandemic week compared to the previous week were physician services [Ontario: -43% (-49%,-37%); BC: -24% (-30%,-19%) (both p<0.001)] and emergency department visits [Ontario: -41% (-47%,-35%); BC: -29% (-35%,-23%) (both p<0.001)]. Hospital admissions declined by 27% (-32%,-23%) in Ontario and 21% (-26%,-16%) in BC (both p<0.001). Resource use subsequently rose but did not return to pre-pandemic levels. Only home care and dialysis clinic visits did not significantly decrease compared to pre-pandemic. Costs for COVID-19 cases represented 1.3% and 0.7% of total direct healthcare costs in 2020 in Ontario and BC, respectively.
Citation: Zeitouny S, Cheung DC, Bremner KE, Pataky RE, Pequeno P, Matelski J, et al. (2023) The impact of the early COVID-19 pandemic on healthcare system resource use and costs in two provinces in Canada: An interrupted time series analysis. PLoS ONE 18(9): e0290646. https://doi.org/10.1371/journal.pone.0290646
Editor: Masoud Behzadifar, Lorestan University of Medical Sciences, ISLAMIC REPUBLIC OF IRAN
Received: February 2, 2023; Accepted: August 11, 2023; Published: September 8, 2023
Copyright: © 2023 Zeitouny et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: In Ontario, the dataset from this study is held securely in coded form at ICES. While legal data sharing agreements between ICES and data providers (e.g., healthcare organizations and government) prohibit ICES from making the dataset publicly available, access may be granted to those who meet pre-specified criteria for confidential access, available at www.ices.on.ca/DAS. The full dataset creation plan and underlying analytic code are available from the authors upon request, understanding that the computer program may rely on coding templates or macros that are unique to ICES and therefore either inaccessible or may require modification. In British Columbia, access to data provided by the Data Steward(s) is subject to approval, but can be requested for research projects through the Data Steward(s) or their designated service providers.
Funding: Funding/support This study was funded by a Canadian Institutes of Health Research (CIHR) operating grant: COVID-19 Rapid Research Funding Opportunity (funding reference number VR4 172774) to Dr. Krahn and Dr. Kulkarni. This study was supported by ICES, which is funded by an annual grant from the Ontario Ministry of Health (MOH) and the Ministry of Long-Term Care (MLTC). This work was also undertaken, in part, thanks to funding from the Canada Research Chairs program to Dr. Krahn, Dr. Walker and Dr. Sander and an Ontario Early Researcher Award to Dr. Wong. Dr. Mendlowitz has received postdoctoral fellowships from the Canadian Network on Hepatitis C (CanHepC) and the Canadian Institutes of Health Research. CanHepC is funded by a joint initiative of the Canadian Institutes of Health Research (NPC-178912) and the Public Health Agency of Canada. The Canadian Centre for Applied Research in Cancer Control is funded by the Canadian Cancer Society.
Competing interests: The authors have declared that no competing interests exist.
Introduction
Healthcare news coverage in the latter part of 2022 was dominated by stories of a system in crisis [1–3]. Worldwide, the COVID-19 pandemic challenged the resilience of health systems, and revealed their strengths and limitations [4]. Analysis of the broad effects of the shock of the COVID-19 pandemic on healthcare services can help health systems prepare for future crises. While Canadian healthcare systems strived to meet the needs of COVID-19 patients, their immediate “ramp-down” strategies [5, 6] created backlogs in non-COVID-19 care [7] and potentially compromised the healthcare, and possibly the health, of many patients [6, 8–11]. In Ontario and British Columbia (BC), two provinces in Canada, non-COVID-19 healthcare services were alternately reduced and restarted as COVID-19 case counts fluctuated in communities [12, 13].
The initial ramp-down started in March 2020 when Canadian healthcare systems began reducing elective procedures and non-emergency care [5, 6, 8], and reconfiguring facilities in preparation for COVID-19 patients [14, 15]. In Canada and internationally, there were reports of major reductions in cancer screening [16–19], diagnosis [20], and treatment [21, 22] in 2020 compared to previous years, as well as lower numbers of hospital admissions [23, 24] and emergency department (ED) visits [10, 11, 25–27]. While some non-COVID-19 care was transitioned from in-person to virtual [28–32] and from inpatient to outpatient [33], concerns about delivering ideal care under COVID-19 circumstances were identified [34–36]. There is a large body of literature focussing on individual disease groups or resource categories, but limited studies have evaluated the costs and healthcare resource use of the entire health system.
Thus, the aim of our study was to assess the initial impact of COVID-19 directives and related mitigation measures on total healthcare costs and resource use across inpatient, outpatient, and long-term care in a concerted fashion for the populations in two provinces within the Canadian universal health system. This broad scope of healthcare services addresses an important information gap by allowing comparisons across resources for a holistic understanding of how the healthcare system reacted and which resources were prioritized within the pandemic response. Specifically, we assessed the changes in the trends of publicly-funded healthcare services use and direct healthcare costs in Ontario and BC.
Methods
Data sources and settings
Our study analyzed healthcare utilization and costs using healthcare administrative datasets that capture publicly-funded outpatient care, inpatient hospital and long-term care, home care services, and prescription drugs for nearly all residents in Ontario and BC. In both provinces, data are collected for administrative purposes by the provincial Ministries of Health, and under the provincial universal health insurance plans [37, 38]. The datasets for Ontario are housed at ICES [38]. These datasets were linked using unique encoded identifiers and analyzed at ICES. Data from the BC Ministry of Health [39–46] were provided through Population Data BC [47]. These datasets were linked using unique encoded identifiers and analyzed by the BC research team. A description of the data sources is in S1 Table in S1 File.
This study was approved by the Research Ethics Boards at the University Health Network in Toronto, Ontario and BC Cancer. ICES is an independent, non-profit research institute whose legal status under Ontario’s health information privacy law allows it to collect and analyze healthcare and demographic data, without consent, for health system evaluation and improvement. In BC, a waiver of consent was granted due to the nature of the data and minimal risk of the study. Guidance on Reporting of studies Conducted using Observational Routinely-collected health Data (RECORD) statement was followed (S1 Checklist in S1 File) [48].
Study design and inclusion criteria.
This was a repeated cross-sectional study from a health system perspective. We examined repeated weekly patient-level measures of healthcare costs and resource use using a micro-costing approach from January 1, 2018, to December 27, 2020, in Ontario and BC separately. In each weekly time interval, defined according to the calendar of the International Organization of Standardization (ISO) [49–51], the study population included all individuals who were alive and registered in the provincial public health plan (Ontario Health Insurance Plan [52] (OHIP) and BC Medical Services Plan [45] (MSP), respectively) for at least one day.
In order to estimate the costs incurred by COVID-19 patients, we identified people with their first positive SARS-CoV-2 test result in each week of 2020 from the Ontario C19INTGR COVID-19 database [53, 54] and the BC Ministry of Health COVID-19 Test Lab Data [46, 55] and followed them until death or December 27, 2020, over two time periods: acute COVID-19, defined as the first 4 weeks (inclusive) after the positive test, and post-acute COVID-19, from the end of the acute COVID-19 period to end of observation [56, 57]. Individuals were in the non-COVID-19 group as long as they did not have a positive SARS-CoV-2 test result.
Study outcomes and statistical analysis
Population characteristics of interest included age, sex, rural/urban residence [58, 59], neighbourhood income quintile, Ontario Marginalization Index quintiles [60, 61] (Ontario only), Ontario Local Health Integration Network (LHIN) [62] or BC Health Authority [63] of residence, and a measure of comorbidity derived from the weighted Aggregated Diagnosis Groups (ADGs) from John’s Hopkins’ Adjusted Clinical Groups® System (ACG®) version 10 [64, 65]. This score, developed using administrative data from Ontario, represents the relative risk of one-year all-cause mortality [64]. Population characteristics as of January 1, 2020 were summarized using means and standard deviations for continuous variables, and frequencies and percentages for categorical variables.
Databases and healthcare services used in our analyses are described in S2 Table in S1 File.
For our analyses of healthcare resource use in each weekly interval, we calculated rates per 10,000 person weeks of acute hospital admissions, emergency department visits, physicians’ services, same day surgery, outpatient prescription drug claims (restricted to prescription drugs covered under the provincial drug benefit plan for eligible individuals in Ontario [66], but including all prescription drugs dispensed in BC irrespective of the insurance payer), laboratory tests including SARS-CoV-2 tests, home care services, and admissions to publicly-funded long-term care (LTC) and complex continuing care (CCC) facilities. Rates of visits to cancer clinics and dialysis clinics were included for Ontario only, as these were not made available in BC. We used standard established methods for estimating healthcare costs from administrative data [67, 68] and we computed the cost of health services used by active registered residents, including hospitalization, emergency department visits, same day surgery, outpatient clinics (Ontario only), physicians’ services, laboratory tests, home and community care, LTC, CCC, and outpatient prescription medications. Costs for all years were expressed in 2020 Canadian dollars (CAD) using Statistics Canada’s Consumer Price Index for healthcare [69].
We generated descriptive analyses of healthcare service use, absolute cost of healthcare services, and rates of utilization and costs per 10,000 person-weeks in each week from 2018 to 2020, in the full population.
The first analysis examined healthcare costs and utilization of select healthcare resources, namely hospital admissions, emergency department visits, physicians’ services, dialysis clinics (Ontario only) and cancer clinics (Ontario only), and home and community care in the full population. Interrupted time series (ITS) analysis was used to estimate changes in the level and trends of the study outcomes, with March 16–22, 2020, when COVID-19 was declared a pandemic in Canada, as the first week of the pandemic [5, 70–72]. ITS models were fit using linear mixed effects regression (LMER) using a Gaussian likelihood on the weekly outcome rates. Visual inspection of the weekly outcome data over the study period informed the specification of the fixed and random effects: for the fixed effects, a linear time trend in study week was estimated for the pre-period, a wild-point indicator was used for the first week of the pandemic, and a quadratic function in week number was used in the pandemic period. Indicator variables for the last week of December, the first week of January, and weeks with other statutory holidays were also included as fixed effects. Each of the above terms was allowed to interact with an indicator variable for province, which afforded separate estimates for each province while fitting only one generalized linear mixed model per outcome. A random intercept for each week of the ISO calendar (i.e., week number within year) was specified as a means of modelling shared seasonal effects and reducing residual autocorrelation [73]. We examined autocorrelation plots both with and without inclusion of the lag-1 outcome as a predictor and determined that its inclusion was generally helpful in ensuring acceptable levels of residual autocorrelation for the final model fits. Alpha of 0.05 was adopted as the threshold for statistical significance. Analyses were conducted using SAS version 9.4 and R version 3.6.1.
A second analysis compared the use of each resource in each week of 2020 to the average weekly use of the resource over both 2018 and 2019, and plotted the percent change in each week of 2020.
A third analysis tabulated healthcare cost estimates in each quarter (consecutive 13 weeks) of 2020 for the full population. The percent change in cost for each resource in each quarter of 2020 compared to its pre-pandemic level was calculated.
A fourth analysis calculated the percentage of the absolute healthcare costs incurred by individuals who tested positive for COVID-19 in 2020 in each province, and segregated by duration since COVID-19 infection, relative to the direct healthcare costs incurred by the full population in 2020.
Results
Description of study populations
In the first week of 2020, the study populations included 14,973,016 individuals in Ontario and 5,123,554 in BC (Table 1). In Ontario, the mean (standard deviation) age was 41.2 (23.0) years, 50.6% of individuals were female, 89.7% of the population resided in urban areas, and 89.1% had a weighted ADG score between -19 and 20. The BC study population had a mean age of 42.2 (23.0) years, 50.6% were female, 88.8% resided in urban areas, and 86.9% had an ADG score between -19 and 20.
Resource use by category
The estimated coefficients of the ITS model are presented in S3 to S6 Tables in S1 File. At the declaration of the pandemic in Canada (March 16 to 22, 2020) as compared to the last pre-pandemic week (March 9 to 15, 2020), the largest level declines (95% confidence interval) in resource use were physician service claims [Ontario: -43% (-49% to -37%), p<0.001; BC: -24% (-30% to -19%), p<0.001] and ED visits [Ontario: -41% (-47 to -35%), p <0.001; BC: -29% (-35% to -23%), p<0.001] (Fig 1a and 1b). These corresponded to immediate level drops of 1,437 (1,259 to 1,616) physician service claims per 10,000 person-weeks in Ontario and 518 (347 to 689) claims per 10,000 person-weeks in BC in the week of March 16, 2020 (S3 Table in S1 File). Similarly, ED visits decreased by 34 (30 to 38) visits per 10,000 person-weeks in Ontario and 17 (12 to 22) visits per 10,000 person-weeks in BC (S4 Table in S1 File). Sharp level declines in hospital admissions were also seen in both provinces [Ontario: -27% (-32% to -23%), p<0.001; BC: -21% (-26% to -16%), p<0.001] (Fig 1c). These corresponded to decreases of 4 (3 to 5) and 4 (3 to 4) admissions per 10,000 person-weeks in Ontario and BC, respectively (S5 Table in S1 File).
Number of a) physician services claims, b) emergency department visits, and c) hospital admissions per 10,000 person-weeks in each ISO week from January 1, 2018 to December 27, 2020 in Ontario and British Columbia (BC). These data were analyzed using interrupted time series (ITS). The vertical line in each graph indicates the first week of the pandemic, March 16 to March 22, 2020.
There was also a decline in visits to cancer clinics in Ontario [-11% (-16% to -6%), p<0.001], corresponding to a decrease of 2 (1 to 3) visits per 10,000 person-weeks (S1 Fig in S1 File). Of the resources analyzed with ITS, only home care services and visits to dialysis clinics did not demonstrate statistically significant decreases in March 2020 [home care p = 0.11 and p = 0.70 in Ontario and BC, respectively; dialysis p = 0.58, Ontario] (S2 Fig in S1 File).
Weekly resource use (by category) in 2020 versus historical averages
Fig 2a and 2b show the overlaid patterns of select resource categories in each week of 2020 normalized to their two-year pre-pandemic average in Ontario and BC, highlighting the different ways the pandemic impacted each resource. In the first pandemic week, same day surgery visits sharply decreased by 69% in Ontario and 48% in BC, compared with the weekly averages of 2018 and 2019, while ED visits and hospital admissions decreased by 28% and 35% in Ontario, respectively, and 31% and 38% in BC. Even with gradual returns toward pre-pandemic levels, many resources notably remained below full capacity by December 2020. In both Ontario and BC, ED visits were still at least 10% (Ontario) and 7% (BC) below pre-pandemic levels from July 2020 to September 2020, only to decrease again for the remainder of 2020.
Percentage change in use per 10,000 person-weeks in 2020 compared with the average weekly use in 2018 and 2019 in (a) Ontario and (b) British Columbia for hospital admissions, same day surgery visits, laboratory tests (do not include SAR-CoV-2 tests), long-term care admissions, outpatient prescription drug claims, and cancer clinic and dialysis clinic visits (Ontario only). This shows the overlaid performance of select resource categories in each ISO week of 2020 normalized to their two-year pre-pandemic average in Ontario and British Columbia. The vertical line indicates the first week of the pandemic, March 16 to March 22, 2020.
The notable exception was a transient increase of up to 48% in the number of prescription drug claims in May and June 2020 in Ontario, coinciding with a change in dispensing policies (limited to a 30-day supply rather than 100-day supply per prescription to limit drug stockpiling). In BC, the number of prescription drug claims remained relatively stable, with only a small increase of 19% in the first week of the pandemic in 2020.
Three distinct patterns were observed: I) a sharp decline and slow recovery, mostly to lower than pre-pandemic levels (as above); II) a saw-tooth pattern in same-day surgery, illustrating the stop-and-start nature of the ramp-down and ramp-up strategies as well as seasonal variability (e.g., statutory holidays); and III) stable resources that fluctuated by a relatively small margin, such as home care in both provinces, prescription drugs in BC, and dialysis and cancer clinics in Ontario.
Healthcare costs
The ITS analyses showed that in the week of March 16 to 23, 2020, total direct healthcare costs declined by 27% (23% to 31%, p<0.001) in Ontario and 18% (14% to 22%, p<0.001) in BC, compared to the final pre-pandemic week. These equate to decreases of $161,407 ($140,009 to $182,805) and $72,689 ($49,126 to $96,253) per 10,000 person-weeks in each province, respectively (Fig 3, S6 Table in S1 File).
These data were analyzed using interrupted time series (ITS). The vertical line indicates the first week of the pandemic, March 16 to March 22, 2020.
As shown in Tables 2 and 3, the total costs for the healthcare services included in this study in the first quarter of 2020 were $10.1 billion in Ontario and $3.2 billion in BC, 6.2% and 2.5% lower than the average quarterly cost in 2018 and 2019, respectively (S7 and S8 Tables in S1 File). The total costs in the second quarter were approximately $8.3 billion in Ontario and $2.8 billion in BC, 23% and 12% lower than pre-pandemic, respectively. In Ontario, total costs increased to $9.8 billion and almost $10.2 billion in the third and fourth quarters, respectively, but the latter was still 5.5% below the average pre-pandemic quarterly cost. In BC, total costs increased in the third and fourth quarters of 2020 to $3.1 billion and almost $3.3 billion respectively, approximately equal to and almost 4% higher than pre-pandemic. Some of the resources for which quarterly costs decreased during the pandemic appeared to return to pre-pandemic levels more than others. In Ontario, costs for same day surgery and physician services (primary care providers) were as high as pre-pandemic levels by the last quarter of 2020. In BC, the costs for same day surgery and services of both specialist and primary care providers were higher than in the pre-pandemic years in the last two quarters of 2020.
For readability, costs are rounded to the nearest thousand dollars; the percentages were calculated using costs estimated to the nearest dollar.
For readability, costs are rounded to the nearest thousand dollars; the percentages were calculated using costs estimated to the dollar.
In both provinces, hospitalization, physician services, and outpatient prescription drugs made up the largest components of total cost, and hospitalization was the main driver for the absolute cost declines in the second quarter of 2020 (Tables 2 and 3). In Ontario, the quarterly costs for visits to cancer and dialysis clinics, outpatient prescription drugs and cancer drugs covered by the New Drug Funding Program [75] were higher in 2020 than in the previous two years.
Direct healthcare costs for COVID-19
The total costs in 2020 for all healthcare services included in this study were approximately $12.4 billion in BC and $38.4 billion in Ontario. During this time, costs for cases with a positive SARS-Cov-2 test from diagnosis to death or censoring were approximately $89.8 million in BC and $504 million in Ontario, representing 0.7% and 1.3% of total costs in BC and Ontario, respectively (S9 Table in S1 File).
Discussion
Our study systematically examined the initial effects of the COVID-19 pandemic on total direct healthcare resource use and costs in two Canadian provinces with universal healthcare, demonstrating the heterogeneous impact of the pandemic across healthcare resources. Early in the pandemic, healthcare resources were reallocated to meet the anticipated and real demands of patients with COVID-19 [5, 6, 8]. However, while we observed immediate decreases in use of physician services, hospital admissions, and emergency department visits, it is reassuring that visits to dialysis clinics, an essential and life-sustaining resource, did not appear to decrease during the first year of the COVID-19 pandemic. Similarly, the rate of home care services did not significantly change, and the decline in visits to cancer clinics in Ontario was smaller than the declines of other resources, suggesting some degree of prioritization of home care and cancer care, although we did not examine the types of services provided.
Several other studies examined changes in cancer care during the COVID-19 pandemic. A study in Ontario found that visits for cancer systemic therapy and radiation therapy were 3.5% and 21% lower, respectively, in the year after March 31, 2020 than in the prior year [77]. Another Canadian study predicted increased cancer-related deaths in the long-term as a result of diagnostic and treatment delays [78]. In England, the mean number of weekly radiation therapy courses delivered by the National Health Service decreased by 20%, 6% and 12% in April, May and June 2020, respectively, compared to the same months in 2019 [22]. The greatest decreases were in treatment for prostate and non-melanoma skin cancers. In both Ontario and England, increased use of hypofractionated radiation regimens explained some of the decreased number of visits [22, 77]. With respect to cancer detection, studies in Japan and Brazil reported reductions in the numbers of mammograms performed and increases in the proportions of patients diagnosed with advanced breast cancer in the pandemic period compared to before [16, 17]. Therefore, there is some evidence from these studies and ours that efforts to maintain cancer treatment were made early in the pandemic, while cancer screening was reduced. The long-term impacts of cancer care delays and deferrals leading to more severe presentations are yet to be revealed [16, 78]. Similar to other studies [26, 27], we found that the rates of ED visits significantly declined at the start of the pandemic compared to the last pre-pandemic week, by 41% in Ontario and 24% in BC, and never returned to pre-pandemic levels in 2020. This is consistent with ED avoidance by people with milder conditions, diversion of human and diagnostic resources away from urgent care, as well as significant decreases in infections associated with lockdown measures. With respect to changes in patterns of emergency medical care, a study in the Canadian city of Calgary, Alberta, reported 35% and 59% fewer patients presenting to adult and pediatric hospital EDs, respectively from December 2019 to June 2020 compared with the same months in the previous two years [26]. Corresponding reductions of 26% to 30% in overall consultations were observed in 20 emergency departments in Germany during the first 3 waves of COVID-19 from March 2020 to June 2021 [27].
Similarly, analyses from 18 hospitals in BC reported decreases of 57% and 70% in visits to pediatric and general hospital EDs, respectively, by children aged 0 to 18 years during the first 6 weeks of the pandemic compared with the same weeks in the previous year [10]. One study in Ontario examined ED and hospital admissions with mental health diagnoses from January 2019 to March 2021, and found sharp decreases of 37% and 30% in rates of ED visits and hospital admissions, respectively, in April 2020 compared with April 2019 [79]. Rates of ED visits for mental health diagnoses increased by July 2020 only to gradually decrease again.
In aggregate, these findings demonstrate that the healthcare system was able to continue providing life-sustaining care to patients who needed it, for example those receiving hemodialysis. However, the use of services appeared to decline according to the severity of conditions they are intended to treat, with greater reductions in the use of elective procedures, and preventive, screening, and routine care than urgent and semi-urgent services [35, 80, 81]. Furthermore, after sharp initial declines in use, recovery was slow for many healthcare resources, and their use did not return to pre-pandemic levels by December 2020. Notably, the recovery was much less than needed to manage the backlogs (areas above the curves with negative values, and below the zero-axis), which raises concerns about the residual unmet healthcare needs of the population. This could be a compounding effect if unmet needs resulted in acute events that could have been avoided with earlier care.
The direct healthcare costs for COVID-19 cases in 2020 corresponded to a modest share of the healthcare costs for the full populations in Ontario and BC, at 1.3% and 0.7%, respectively. The magnitude of the resources used by COVID-19 patients was far outweighed by reduced service utilization in the general population. However, the economic impact of COVID-19 includes costs for vaccines, personal protective equipment, improving indoor air quality, paid sick leave, and other support beyond the direct healthcare costs measured in our study [82, 83]. Furthermore, much of the human burden of COVID-19 and long COVID-19 have not yet been estimated [84], and may be beyond our metrics.
In anticipation of future health crises, it is critical to ensure an equitable allocation of resources and a “reserve” within the healthcare system. A resilient healthcare system must respond to the needs of COVID-19 patients while preserving standards of care for all patients, and preserving access to care for its community [85]. Resilient hospitals and healthcare systems must implement changes that will be able to meet any challenges they may encounter and that will be useful even during normal times [85]. Our data suggested that such a reserve was not present within the Canadian universal healthcare system. In planning for COVID-19 recovery phases, integration and communication among policy makers, health economists, and clinicians across services may be necessary to help the healthcare system adapt to crises [86], inform resource allocation, and prioritize recovery efforts.
Strengths and limitations
Strengths of this study include the use of comprehensive population-wide data which included almost all healthcare resource use in two populous Canadian provinces with provincial publicly-funded healthcare. This study used standardized methods to estimate resource use and costs [87, 88], ensuring valid outcome measures. However, some of the absolute differences in the service rates between provinces may be due to differences in the scope of the data available in Ontario and BC and underlying program structure.
Despite the comprehensiveness of our patient-level data, we were not able to measure the unmet healthcare needs of those who did not access resources. Furthermore, we could not capture the expenditures related to health systems, such as costs associated with infrastructure changes or the rapid adoption of telehealth initiatives. While health system directives were a major factor in the observed changes, some of the reductions in healthcare use may also have been driven by patients’ avoidance of medical settings [80]. However, social distancing behaviours, directives, or patients’ motivation are likely to have acted as mediators, rather than confounders, of the changes in healthcare costs and use that we observed. That is, the behavioural changes and policies are part of the COVID-19 pandemic effect, rather than confounding the true effect.
Although the ITS analysis allowed us to project a counterfactual (based on pre-pandemic secular trends had the pandemic not occurred), we cannot comment on the appropriateness of this benchmark. However, this is likely to represent a conservative bias as a return to pre-COVID-19 trends and levels does not account for the backlog accrued during COVID-19. Our study covered the immediate responses of the healthcare system to the shocks of the first two waves of COVID-19. Subsequent time periods with waves of new COVID-19 variants and relaxation of public health measures would require different methods to answer different questions. Finally, even though the experiences in Ontario and BC may not represent those in all of Canada or other countries, their similar patterns suggest that other provinces might have responded in much the same way to the initial challenges of the COVID-19 pandemic, and may serve as a blueprint for other publicly-funded healthcare systems in the world.
In summary, our study confirms that the overall use of physician services, hospital admissions, and emergency department visits greatly decreased in response to the initial COVID-19 pandemic, while life-sustaining dialysis care was maintained, as were home care services and cancer care in Ontario. Thus, resource optimization appeared to be appropriate to the degree of necessary care. However, recovery for many healthcare resources was slow after sharp declines, and the resultant backlogs raise concerns about residual unmet needs. Finally, in Ontario and BC, the direct healthcare costs for COVID-19 cases in 2020 represented were outweighed by reduced costs in the general population consequent to decreased resource use.
Conclusions
We observed decreased costs and rates of use for most healthcare resources in two Canadian provinces with predominantly publicly-funded healthcare following the declaration of the COVID-19 pandemic and the associated initial mitigation measures. These changes may have compromised the care of many non-COVID-19 patients in efforts to prepare for the anticipated demand of care for patients with COVID-19. These findings highlight the need for more flexible and sustainable systems that can ensure timely access to adequate care for all individuals. Monitoring to detect and immediately address backlogs could avoid the need for supra-normal levels of activity to "catch up" the system. Further research on the potential consequences of delayed and cancelled medical care will be necessary to mitigate risks in access and quality of healthcare services, address backlog recovery, and avoid healthcare system collapse.
Acknowledgments
We acknowledge the contributions of Dr. Murray Dale Krahn, who passed away on July 1, 2022. Soon after COVID-19 was declared a pandemic, Dr. Krahn foresaw the potential impact that it might have on our healthcare system, and realized the importance of examining system-wide responses in these extraordinary times. He asked not only how the health system would care for patients with COVID-19, but also what would happen to other patients, when healthcare pivoted to prioritize COVID-19. This study is one product of his inspiration, guidance, and pioneering forays into health services research over the years, and we dedicate this paper to his honour.
This document used data adapted from the Statistics Canada Postal CodeOM Conversion File Plus Version 7B, which is based on data licensed from Canada Post Corporation, and/or data adapted from the Ontario Ministry of Health Postal Code Conversion File, which contains data copied under license from ©Canada Post Corporation and Statistics Canada. Parts of this material are based on data and information compiled and provided by Ontario Ministry of Health (MOH). The analyses, conclusions, opinions and statements expressed herein are solely those of the authors and do not reflect those of the funding or data sources; no endorsement is intended or should be inferred. Parts of this material are based on data and information provided by Ontario Health (OH). The opinions, results, view, and conclusions reported in this paper are those of the authors and do not necessarily reflect those of OH. No endorsement by OH is intended or should be inferred. Parts of this material are based on data and/or information compiled and provided by CIHI and the Ontario Ministry of Health. The analyses, conclusions, opinions and statements expressed herein are those of the authors and do not reflect those of the funding or data sources; no endorsement is intended or should be inferred. Parts of this material are based on data adapted from Statistics Canada, the Canada Census, 2016. This does not constitute an endorsement by Statistics Canada of this product. We thank IQVIA Solutions Canada Inc. for use of their Drug Information File. We thank the Toronto Community Health Program Profiles Partnership for providing access to the Ontario Marginalization Index.
References
- 1.
Hahn P. Health care surpasses inflation as top national issue of concern: Nanos. CTV News. December 1, 2022. https://wwwctvnewsca/health/health-care-surpasses-inflation-as-top-national-issue-of-concern-nanos-16176739.
- 2.
Moir M, Barua B. Overwhelming evidence—it’s time to fix Canadian health care. Published on Fraser Institute. September 9, 2022. https://wwwfraserinstituteorg/article/overwhelming-evidence-its-time-to-fix-canadian-health-care
- 3.
Tasker JP. ’This is a crisis’: Head of medical association warns that the health-care system faces ’collapse’. CBC News. September 21, 2022. https://wwwcbcca/news/politics/canadian-health-care-system-collapse-16590461.
- 4. Haldane V, De Foo C, Abdalla SM, Jung A-S, Tan M, Wu S, et al. Health systems resilience in managing the COVID-19 pandemic: lessons from 28 countries. Nat Med. 2021;27:964–80. pmid:34002090
- 5.
Angus H, Williams D, Anderson M. Ramping down elective surgeries and other non-emergent activities. March 15, 2020. https://www.health.gov.on.ca/en/pro/programs/publichealth/coronavirus/docs/memos/DM_OH_CMOH_memo_COVID19_elective_surgery_March_15_2020.pdf.
- 6. Thiara S, Henry B, Patrick D, Kanji H. British Columbia’s COVID-19 experience. BC Med J. 2020;62(8):277–9.
- 7. Wang J, Vahid S, Eberg M, Milroy S, Milkovich J, Wright FC, et al. Clearing the surgical backlog caused by COVID-19 in Ontario: a time series modelling study. CMAJ. 2020;192:E1347–E56. pmid:32873541
- 8. Archer SL. Providing care for the 99.9% during the COVID-19 pandemic: How ethics, equity, epidemiology, and cost per QALY inform healthcare policy. Healthc Manage Forum. 2020;33(5):239–42. pmid:32638633
- 9. Bronskill SE, Maclagan LC, Maxwell CJ, Iaboni A, Jaakkimainen L, Marras C, et al. Trends in health service use for Canadian adults with dementia and Parkinson disease during the first wave of the COVID-19 pandemic. JAMA Health Forum. 2022;3(1):e214599. pmid:35977228
- 10. Goldman RD, Grafstein E, Barclay N, Irvine MA, Portales-Casamar E. Paediatric patients seen in 18 emergency departments during the COVID-19 pandemic. Emerg Med J. 2020;37:773–7. pmid:33127743
- 11. Grunau B, Helmer J, Acker J, Deakin J, Armour R, Tallon J, et al. Decrease in emergency medical services utilization during early stages of the COVID-19 pandemic in British Columbia. CJEM. 2021;23(2):237–41. pmid:33709367
- 12.
BC Ministry of Health. A commitment to surgical renewal in B.C. Spring—Fall. May 7, 2020. https://www2.gov.bc.ca/assets/gov/health/conducting-health-research/surgical-renewal-plan.pdf
- 13.
Ontario Health A Measured Approach to Planning for Surgeries and Procedures During the COVID-19 Pandemic. June 15, 2020 updated). https://www.ontariohealth.ca/sites/ontariohealth/files/2020-05/A%20Measured%20Approach%20to%20Planning%20for%20Surgeries%20and%20Procedures%20During%20the%20COVID-19%20Pandemic.pdf.
- 14. Brown DR, Hennecke P, Nottebrock D, Dhillon P. Vancouver Convention Health Centre (COVID-19 Response): Planning, Implementation, and four lessons learned. Am J Disaster Med. 2020;15(2):143–8. pmid:32804395
- 15. Patey C, Asghari S, Norman P, Hurley O. Redesign of a rural emergency department to prepare for the COVID-19 pandemic. CMAJ. 2020;192:E518–E20. pmid:32317277
- 16. Adachi K, Kimura F, Takahashi H, Kaise H, Yamada K, Ueno E, et al. Delayed diagnosis and prognostic impact of breast cancer during the COVID-19 pandemic. Clin Breast Cancer. 2023;23(3):265–71. pmid:36717319
- 17. Solla Negrao EM, Cabello C, Conz L, Mauad ED, Zeferino LC. The impact of the COVID-19 pandemic on breast cancer screening and diagnosis in a Brazilian metropolitan area. J Med Screen. 2023;30(1):42–6. pmid:36071628
- 18. Bakouny Z, Paciotto M, Schmidt AL, Lipsitz AL, Choueiri TK, Trinh Q-D. Cancer screening tests and cancer diagnoses during the COVID-19 pandemic. JAMA Oncology. 2021;7(3):458–60. pmid:33443549
- 19. Liu J. COVID-19 update to BC cancer sceeening availability. BC Med J. March 25, 2020. Available from: https://bcmj.org/blog/covid-19-update-bc-cancer-screening-program-availability.
- 20. Dinmohamed AG, Visser O, Verhoeven RHA, Louwman MWJ, van Nederveen FH, Willems FM, et al. Fewer cancer diagnoses during the COVID-19 epidemic in the Netherlands. Lancet Oncol. 2020;21(603):750–1.
- 21. Eskander A, Li Q, Hallet J, Coburn N, Hanna TP, Irish J, et al. Access to cancer surgery in a universal health care system during the COVID-19 pandemic. JAMA Network Open. 2021;4(3):e211104. pmid:33704472
- 22. Spencer K, Jones CM, Girdler R, Roe C, Sharpe M, Lawton S, et al. The impact of the COVID-19 pandemic on radiotherapy services in England, UK: a population-based study. Lancet Oncol. 2021;22:309–20. pmid:33493433
- 23. Blecker S, Jones SA, Petrilli CM, Admin AJ, Weerahandi H, Francois F, et alI. Hospitalizations for chronic disease and acute conditions in the time of COVID-19. JAMA Intern Med. 2020;181(2):269–71.
- 24. Pelletier JH, Rakkar J, Au AK, Fuhrman D, Clark RS, Horvat CM. Trends in US pediatric hospital admissions in 2020 compared with the decade before the COVID-19 pandemic. JAMA Network Open. 2021;4(2):e2037227. pmid:33576819
- 25. Jeffery MM, D’Onofrio G, Paek H, Platts-Mills TF, Soares WE, Hoppe JA, et al. Trends in emergency department visits and hospital admissions in health care systems in 5 states in the first months of the COVID-19 pandemic in the US. JAMA Intern Med. 2020;180(10):1328–33. pmid:32744612
- 26. Lane DJ, Blanchard IE, Buick JE, Shaw M, McRae AD. Changes in presentation, presenting severity, and disposition among patients accessing emergency services during the first months of the COVID-19 pandemic in Calgary, Alberta: a descriptive study. CMAJ Open 2021;9(2):E592–E601. pmid:34074633
- 27. Schranz M, Boender TS, Greiner T, Kocher T, Wagner B, Greiner F, et al. Changes in emergency department utilisation in Germany before and during different phases of the COVID-19 pandemic, using data from a national surveillance system up to June 2021. BMC Public Health. 2023;23:799. pmid:37131165
- 28. Glazier RH, Green ME, Wu FC, Frymiore E, Kopp A, Kiran T. Shifts in office and virtual primary care during the early COVID-19 pandemic in Ontario, Canada. CMAJ. 2020;193:E200–E10.
- 29. Gosse PJ, Kassardjian CD, Masellis M, Mitchell SB. Virtual care for patients with Alzheimer disease and related dementias during the COVID-19 era and beyond. CMAJ. 2021;193:E371–E7. pmid:33722828
- 30. Roberts A, Johnston GHF, Landells C. COVID-19: pivoting from in-person to virtual orthopedic surgical evaluation. Can J Surgery. 2021;64(1):E101–E2. pmid:33599451
- 31. Yang CL, Waterson S, Eng JJ. Implementation and evaluation of the Virtual Graded Repetitive Arm Supplementary Program (GRASP) for individuals with stroke during the COVID-19 pandemic and beyond. Physical Therapy. 2021;101(6):1–9. pmid:33682872
- 32. Connor MJ, Winkler M, Miah S. COVID-19 pandemic—is virtual urology clinic the answer to keeping the cancer pathway moving [comment]. BJU Int. 2020;125:E3–E4.
- 33. Gleicher Y, Peacock S, Peer M, Wolfstadt J. Transitioning to outpatient arthroplasty during COVID-19; time to pivot. CMAJ. 2021;193:E455. pmid:33782174
- 34. Nicholas DB, Belletrutti M, Dimitropoulos G, Katz SL, Rapoport A, Urschel S, et al. Perceived impacts of the COVID-19 pandemic on pediatric care in Canada: A roundtable discussion. Glob Pediatr Health. 2020;7:1–10.
- 35. Muellers KA, Andreadis K, Ancker JS, Horowitz CR, Kaushal R, Lin JJ. Provider and patient experiences of delays in primary care during the early COVID-19 pandemic. J Healthc Quality 2023;45(3):169–76. pmid:37010342
- 36. Noceda AVG, Acierto LMM, Bertiz MCC, Dionisio DEH, Laurito CBL, Sanchez GAT, et al. Patient satisfaction with telemedicine in the Philippines during the COVID-19 pandemic: a mixed methods study. BMC Health Serv Res. 2023;23:277. pmid:36949479
- 37. Lucyk K, Lu M, Sajobi T, Quan H. Administrative health data in Canada: lessons from history. BMC Med Inform Dec Making. 2015;15:69. pmid:26286712
- 38. Schull MJ, Azimaee M, Marr M, Cartagena RG, Vermeulen MJ, Ho M, et al. ICES: Data, Discovery, Better Health. Int J Pop Data Sci. 2020;4(2):14. pmid:32935037
- 39.
British Columbia Ministry of Health [creator] (2020). Home & Community Care (Continuing Care). V2. Population Data BC [publisher]. Data Extract. MOH (2020). from http://www.popdata.bc.ca/data.
- 40.
British Columbia Ministry of Health [creator] (2020). Medical Services Plan (MSP) Payment Information File. V2. Population Data BC. Data Extract. MOH (2020). from http://www.popdata.bc.ca/data.
- 41.
British Columbia Ministry of Health [creator] (2020). PharmaNet. V2. BC Ministry of Health [Publisher]. Data Extract. Data Stewardship Committee (2020). from http://www.popdata.bc.ca/data.
- 42.
British Columbia Ministry of Health [creator] (2020). Vital Events Deaths. Population Data BC [Publisher]. Data Extract. MOH (2020). from http://www.popdata.bc.ca/data.
- 43.
Canadian Institute for Health Information [creator] (2020). Discharge Abstract Database (Hospital Separations). V2. Population Data BC [Publisher]. Data Extract. MOH (2020). from http://www.popdata.bc.ca/data.
- 44.
Canadian Institute for Health Information [creator] (2020). National Ambulatory Care Reporting System. V2. Population Data BC. Data Extract. MOH (2020). from http://www.popdata.bc.ca/data.
- 45.
British Columbia Ministry of Health [creator]. Consolidation file (MSP Registration and Premium Billing). V2. Population Data BC [publisher]. Data Extract. MOH (2020). from http://www.popdata.bc.ca/data.
- 46.
British Columbia Ministry of Health [creator]. Population Data BC [publisher]. BC COVID-19 Test Lab Data. Data Extract. MOH (2020). from http://www.popdata.bc.ca/data.
- 47. Ark TK, Kesselring S, Hills B, McGrail KM. Population data BC: Supporting population data science in British Columbia. Int J Pop Data Science. 2020;4(2):1133. pmid:32935036
- 48. Benchimol EI, Smeeth L, Guttmann A, Harron K, Moher D, Petersen I, et al. The REporting of studies Conducted using Observational Routinely-collected health Data (RECORD) Statement. PLoS Med. 2015;12(10): e1001885. pmid:26440803
- 49.
EpochConverter. Week numbers for 2020. https://www.epochconverter.com/weeks/2020.
- 50.
EpochConverter. Week numbers for 2019. https://www.epochconverter.com/weeks/2019.
- 51.
EpochConverter. Week numbers for 2018. https://www.epochconverter.com/weeks/2018.
- 52.
Health care in Ontario. Published June 16, 2017; updated February 3, 2023. https://www.ontario.ca/page/health-care-ontario.
- 53.
Ontario Health. Appendix 1: Case Definitions and DiseaseSpecific Information- Disease: Diseases caused by a novel coronavirus, including Coronavirus Disease 2019 (COVID-19), Severe Acute Respiratory Syndrome (SARS) and Middle East Respiratory Syndrome (MERS). https://wwwhealthgovonca/en/pro/programs/publichealth/coronavirus/docs/2019_case_definitionpdf. Effective June 2022.
- 54.
Ontario Agency for Health Protection and Promotion (Public Health Ontario). Weekly Epidemiologic Summary. COVID-19 in Ontario: Focus on December 20, 2020 to December 26, 2020. Queen’s Printer for Ontario, Toronto, ON. https://filesontarioca/moh-covid-19-weekly-epi-report-en-2020-12-29pdf.
- 55.
Population Data BC. Coronavirus Disease (COVID-19) Case Definition and Related Coding. 2020. https://www.popdata.bc.ca/COVID-19_case_definition.
- 56. Greenhalgh T, Knight M, A’Court C, Buxton M, Husain L. Management of post-acute covid-19 in primary care. BMJ. 2020;370:m3026. pmid:32784198
- 57. Ladds E, Rushforth A, Wieringa S, Taylor S, Rayner C, Husain L, et al. Persistent symptoms after Covid-19: qualitative study of 114 “long Covid” patients and draft quality principles for services. BMC Health Serv Res. 2020;20:1144. pmid:33342437
- 58.
Mechanda K, Puderer H. How postal codes map to geographic areas. Statistics Canada Geograohy Working Paper Series, Catalogue no. 92F0138MIE—No. 001 ISSN: 1481-174X; ISBN: 978-0-662-44674-3. February 12, 2007. https://www150.statcan.gc.ca/n1/en/pub/92f0138m/92f0138m2007001-eng.pdf?st=J28UXCVZ
- 59.
du Plessis V, Besgiri R, Bollman RD. Definitions of “Rural”. Agriculture and Rural Working Paper Series Working Paper No. 61. Statistics Canada, Agriculture Division. December 2002.
- 60.
Matheson F, van Ingen T. Ontario marginalization index: user guide. St Michael’s Hospital; Joint publication with Public Health Ontario Toronto, Ontario. 2018.
- 61. Zygmunt A, Tanuseputro P, James P, Lima I, Tuna M, Kendall CA. Neighbourhood-level marginalization and avoidable mortality in Ontario, Canada: a population-based study. Can J Public Health. 2020;111(2):169–81. pmid:31828730
- 62.
2015 Annual Report of the Office of the Auditor General of Canada. 2015. Chapter 3, Section 3.08 LHINs—Local Health Integration Networks. https://www.auditor.on.ca/en/content/annualreports/reportsbytopic/bytopic_health.html):https://www.auditor.on.ca/en/reports_en/en15/3.08en15.pdf.
- 63.
Government of British Columbia. Regional Health Authorities. updated June 9, 2021. https://www2.gov.bc.ca/gov/content/health/about-bc-s-health-care-system/partners/health-authorities/regional-health-authorities.
- 64. Austin PC, von Walraven C. The mortality risk score and the ADG score: two points-based scoring systems for the Johns Hopkins aggregated diagnosis groups to predict mortality in a general adult population cohort in Ontario, Canada. Med Care. 2011;49:940–7. pmid:21921849
- 65.
The Johns Hopkins ACG system. State of the Art Technology and a Tradition of Excellence in One Integrated Solution. White Paper- General. December 2012. https://www.johnshopkinssolutions.com/wp-content/uploads/2014/04/ACG-White-Paper-Technical-Dec-2012-rev.pdf.
- 66.
Ontario Health. Get coverage for prescription drugs. Published September 20, 2016; updated May 31, 2022. https://wwwontarioca/page/get-coverage-prescription-drugs.
- 67.
Jacobs P, Yim R. Using Canadian administrative databases to derive economic data for health technology assessments. Canadian Agency for Drugs and Technologies in Health. 2009. Ottawa, ON.
- 68.
Wodchis WP, Bushmeneva K, Nikitovic M, McKillop I. Guidelines on Person-Level Costing Using Administrative Databases in Ontario. Health System Performance Research Network. 2013;1:71.
- 69.
Statistics Canada. Table 18-10-0005-01 Consumer Price Index, annual average, not seasonally adjusted. Available from: https://www150.statcan.gc.ca/t1/tbl1/en/cv.action?pid=1810000501.
- 70.
Nielson K. A timeline of COVID-19 in Ontario. Global News. April 24, 2020; updated May 10, 2021;https://globalnews.ca/news/6859636/ontario-coronavirus-timeline/.
- 71.
BC Gov News. Information bulletin. B.C. COVID-19 response update. Victoria. March 19, 2020. https://news.gov.bc.ca/releases/2020HLTH0013-000526.
- 72.
Canadian Institute for Health Information. Canadian COVID-19 intervention timeline. 2022. https://wwwcihica/en/canadian-covid-19-intervention-timeline.
- 73. Wagner AK, Soumerai SB, Zhang F, Ross-Degnan D. Segmented regression analysis of interrupted time series studies in medication use research. J Clin Pharm Ther. 2002;27:299–309. pmid:12174032
- 74.
Ontario Health. What OHIP covers. Publsished April 21, 2017; updated October 21, 2021. https://wwwontarioca/page/what-ohip-covers.
- 75.
Cancer Care Ontario. The New Drug Funding Program. https://www.cancercareontario.ca/en/Funding/New_Drug_Funding_Program.
- 76. Evans WK, Nefsky M, Pater J, Browman G, Cowan DH. Cancer Care Ontario’s New Drug Funding Program: controlled introduction of expensive anticancer drugs. Chron Dis Can. 2002;23(4):152–6. pmid:12517323
- 77. Walker MJ, Wang J, Mazuryk J, Skinner S-M, Meggetto O, Ashu E, et al. Delivery of cancer care in Ontario, Canada, during the first year of the COVID-19 pandemic. JAMA Network Open 2022;5(4):E228855. pmid:35467731
- 78. Malagon T, Yong JHE, Tope P, Miller WH Jr, Franco EL, McGill Task Force on the Impact of COVID-19 on Cancer Control and Care. Predicted long-term impact of COVID-19 pandemic-related care delays on cancer mortality in Canada. Int J Cancer. 2022;150(8):1244–54.
- 79. Saunders NR, Toulany A, Deb B, Strauss R, Vigod SM, Guttmann A, et al. Acute mental health service use following onset of the COVID-19 pandemic in Ontario, Canada: a trend analysis. CMAJ Open. 2021;9(4):E988–E97. pmid:34785528
- 80. Czeisler ME, Marynak K, Clarke KEN, Salsh Z, Shakya I, Thierry JM, et al. Delay or avoidance of medical care because of COVID-19-related concerns- United States, June 2020. MMWR. 2020;69(36):1250–7. pmid:32915166
- 81. Dickinson JA, Thériault G, Singh H, Szafran O, Grad R. Rethinking screening during and after COVID-19. Should things ever be the same again? Can Fam Physician. 2020;66:571–9.
- 82.
Government of Canada Department of Finance Canada. Economic and Fiscal Update 2021. Copyright Her Majesty the Queen in Right of Canada (2021). ww.Canada.ca/Economic-Fiscal-Update.
- 83.
Statistics Canada. COVID-19 in Canada: A two-year update on social and economic impacts. March 10. 2022, Catalogue no. 11-631-X. ISBN 978-0-42200-8. https://www150statcangcca/n1/pub/11-631-x/11-631-x2022001-enghtm.
- 84. Holly E. Tallying the health toll of COVID-19. Nature. 2022;605:410–3.
- 85. Barbash IJ, Kahn JM. Fostering hospital resilience—lessons from COVID-19. JAMA. 2021;326(8):693–4. pmid:34323923
- 86. Legido-Quigley H, Asgari N, Teo YY, Leung GM, Oshitani H, Fukuda K, et al. Are high-performing health systems resilient against the COVID-19 pandemic? The Lancet. 2020;395:848–50.
- 87. de Oliveira C, Pataky R, Bremner KE, Rangrej J, Chan KKW, Cheung WY, et al. Estimating the cost of cancer care in British Columbia and Ontario: A Canadian inter-provincial comparison. Healthc Policy. 2017;12(3):e95–e108.
- 88. McBride ML, de Oliveira C, Duncan R, Bremner KE, Liu N, Greenberg ML, et al. Comparing childhood cancer care costs in two Canadian provinces. Healthc Policy. 2020;15(3):76–88. pmid:32176612