An Observational Study of a National Referral System: An Analysis of 14,266 Referrals in Sierra Leone

Background Referral systems are critical to a well-functioning health system. In 2017, a special cadre of referral coordinators were trained and deployed at every district and tertiary hospital in Sierra Leone. We analyse the referrals coordinated by the network to understand health service utilisation, referral pathways and outcomes. Methods A retrospective observational study of incoming referrals to all district and tertiary hospitals in Sierra Leone from 1 st November 2017 until 31 st October 2018. Multivariate analysis was performed on all referrals and a subgroup analysis of urgent referrals (n=10,865). Hospital preparedness and readiness scores were sourced from the Service Availability and Readiness Assessment 2017. Results 14,266 referrals were captured over the 12 months. Referral indices ranged from 0·51-5·97, with the highest indices found in Freetown and Pujehun. Bed occupancy ranged from 36·8-83·3%. 606 deaths were recorded, mortality rate per referral was 4.25%, with the majority of deaths, 446 (73.60%) occurring in the U5 population. Higher OR for mortality are seen from referrals originating from higher levels of the health system. The OR for mortality for a referral from the lowest level, MCHP, was 0.77, whilst from tertiary level was 2.40 (MCHP 0·77, CHP 0·90, CHC 1·40, district 2·06 and tertiary 2·40). For urgent referrals, factors associated with mortality were U5s, adult non-maternity cases and being seen by a clinician within one hour. Referrals from district hospitals to tertiary hospitals only accounted for 0·5% of all referrals. No correlation was found between referral index, bed occupancy and hospital service availability and readiness scores. The study did not detect any difference in total number of national referrals during the rainy season. Conclusions Wide variations in bed occupancy and referrals by district highlight disparities in health service utilisation. Low rates of referral from district hospitals to tertiary care should be further investigated. Referral indices and referral mortality rates are useful to monitor over time and to make inter district comparisons. Referral systems and pathways are useful indicators of health service utilisation

Conclusions Wide variations in bed occupancy and referrals by district highlight disparities in health service utilisation. Low rates of referral from district hospitals to tertiary care should be further investigated. Referral indices and referral mortality rates are useful to monitor over time and to make inter district comparisons. Referral systems and pathways are useful indicators of health service utilisation and further research should be undertaken to standardise de nitions and identify performance indicators for referral systems in low resource settings.

Background
Referral systems are critical to a well functioning health system. 1 The recent Lancet commission for High Quality Healthcare argues for improved referral systems as a key step to delivering High-Quality health systems. 2 Referrals systems are essential to; ensure patients receive timely appropriate care; increase access to specialist services; increase uptake of primary healthcare and optimise hospital utilisation 3 . Presently, referrals systems in Low and Middle Income Countries (LMICs) and the interface between different levels of the health system are both under-utilised 4 and under-researched. 5 6 7 Whilst access to care is regularly described as a key issue in LMIC health systems, formal referral systems are rare, standardised data on how patients are referred through the health system is sparse and there is little evidence on interventions to improve referral 8 . In this paper we describe the development of a National Referral Service in Sierra Leone and analyse 12 months of referral data. We aim to understand how referrals differ by district, the basic demographics of referred patients, which levels of the health care system patients are being referred from and which variables are associated with mortality.
Sierra Leone has a fragile health system with some of the worst health outcomes in the world, under 5 (U5) mortality is estimated as 156/1000 live births and the maternal mortality ratio as 1165/100,000 live births. 9 The Sierra Leone health system is organised into the following levels; 616 Maternal and Child Health Posts (MCHP); 320 Community Health Posts (CHP); 227 Community Health Centres (CHC) all providing various aspects of primary health care; 14 District and regional Hospitals providing Secondary care and Comprehensive Emergency Obstetric and Newborn care; 4 Tertiary Referral Hospitals, all based in the Western Area province, providing specialist care. 10 The level of clinical service that should be delivered at each facility type is clearly laid out in the Basic Package of Essential Healthcare (BPEHS), 11 however this ambitious national standard is not currently being delivered. Intermittent large scale surveys demonstrate that many health facilities are unable to deliver the BPEHS, with an average health facility service readiness index of 56% 12 .
Challenges with health system inputs including weak supply chains 12 and limited health workforce 13 14 mean that service availability is not constant but in ux and highly dependent on time and location. This contributes to underutilization of available services, 12 frequent bypassing of levels of care 15 and a loss of trust when reported services are not available 16 . Prior to 2017 there was no formal referral system to map the changing service availability landscape and ensure patients were sent to the correct facility and at the correct time to receive care.
The 2017 Sierra Leone Health Sector Strategic Plan highlighted that there is "not yet a functional system of referral to move patients through various levels of care". 10 To address this, the Ministry of Health and Sanitation (MoHS), developed the National Referral Service (NRS) in September 2017. The aims of the NRS are to support and direct patient referrals between health facilities, map national service availability and utilisation in real time, and to develop a communication feedback loop between different levels of the health system. A pre-existing network of referral coordinators who originally supported access to free healthcare for Ebola survivors under the Comprehensive Package of Ebola Survivors programme, was repurposed to support referrals for the general population. The NRS comprises of a network of 18 referral coordinators(RCs), located in every district government hospital and tertiary hospital nationwide, supported by a small support team based in the Ministry of Health and Sanitation. The RCs come from a nursing or community health o cer background and undergo a modular training package focusing on referral concepts, de nitions, basic data collection competencies, communication skills, and patient advocacy, followed with ongoing mentorship and coaching from a central hub in Freetown. The network is in daily communication through Whatsapp, SMS, phone and email. Standardised information is shared daily at the facility level, and monthly at the district and national level.

Methods
For the period of 1st November 2017 until 31st October 2018, all incoming referrals were recorded by referral coordinators nationwide. RCs were noti ed of incoming referrals by telephone call from sending facility, direct patient contact following their referral, or through patient identi cation as a referral upon arrival at destination facility. Patients were followed up during their hospital admission to record process data (referral time, arrival time and seen by clinician time) and in hospital outcomes (did not arrive, discharge, onward referral or death). Referrals were coded as urgent or non-urgent based on the referring clinician's judgement.
Referral data is recorded by referral coordinators on to a standardised paper case report form and then transcribed into an EpiInfo™ datasheet. Data quality checks are performed on the dataset monthly by the NRS support unit in Freetown. Bed occupancy data is collected at 0800 every 24 hours by the RC on a standard template, reported weekly by SMS, and compiled into a Microsoft Excel database.
Population data was obtained from the Sierra Leone Population and Housing census 2015. 17 Health facility data such as health facility density; inpatient bed density; general service readiness; diagnostic capacity; skilled healthcare worker density was obtained from the Service Availability and Readiness (SARA) survey 2017. 12 A basic referral index was calculated for each district hospital = (number of annual referrals/district population) x 1000.
All analysis was performed in Stata v13, StataCorp LP. Pearson's Correlation coe cient was used to evaluate relationships between two continuous variables. Multivariate binomial regression analysis was performed on the subgroup of urgent referrals and odds ratios for mortality was calculated. Variables were selected based on their signi cance in univariate analysis (p < 0·20).
Ethical approval was granted from Sierra Leone Ethics and Scienti c Review Committee.

Results
From 1st November 2017 until 31st October 2018, the NRS dataset recorded 14,266 incoming referrals nationwide ( Table 1). The mean referral index was 1·72, ranging from 0·51 − 5·97, with the highest referral indices seen in Pujehun district and Western Area district. Maternity cases comprised 7215 (50·6%) referrals, U5 referrals accounted for 5594 (39·2%) referrals and the remaining adult population 1457 (10·2%) of all referrals. 10,401 (72·9%) referrals were female. Of all 14,266 referrals, 606(4·2%) died in hospital. The majority of deaths (73·60%) were in the U5 population. 10% of deaths were in the non free health care population. The referral system captured 67 deaths in pregnant women and 25 post-partum deaths.  Monthly referrals ranged from 977 in January to 1297 in May, there was no statistically signi cant difference in number of referrals or mortality by month, including a separate rainy (June-August)/dry season analysis (Fig. 1).. Mean percentage bed occupancy was 56·1% and ranged from 36·8-83·3% between hospitals. Average national bed occupancy did not vary signi cantly by month. District hospital bed occupancy and referral index had no statistically signi cant relationship with; health facility density; inpatient bed density; general service readiness; diagnostic capacity or skilled healthcare worker density as reported by the 2017 Service Availability and Readiness Assessment (appendix 1). The dataset provided facility of origin type, and referral destination facility type for 14,258 referrals (Fig. 2)

Limitations
There are several limitations to this research. Firstly, the NRS dataset is not a comprehensive record of all nationwide referrals. It only attempts to capture incoming referrals to district and tertiary facilities, it does not record referrals occurring between lower levels of health facilities. Additionally, many patients may arrive at secondary and tertiary facilities having previously visited primary healthcare facilities but with no documented record or phone call of their visit. Our study only included referrals if the referring facility made a referral phone call or if a referral note was written. Finally, many patients will not utilise formal referral systems and will bypass lower levels of care 4 , we are unable to quantify this and therefore self-referrals are not included in our study.
Outcome data and data on time to be seen at facility is likely not generalisable to the wider population attending these hospitals, as a principal role of the referral coordinators was to advocate for timely care for incoming referrals, patients not utilising this referral mechanism may experience different delays, quality of care and outcomes. Due to a lack of standardised hospital data collection processes, we were unable to validate the National Referral Database against other MoHS datasets. Additionally, our study was not designed to evaluate whether referrals were appropriate according to the patient condition, diagnosis or facility level within the health system.

Discussion
The rst nationwide analysis of referrals in Sierra Leone demonstrates wide variation between districts in terms of percentage bed occupancy, total number of referrals, range of referral population, referral indices and mortality rate per referral. Two locations have much higher referral indices than the national mean, Pujehun and the Western Area. Pujehun as a district has bene ted from a targeted health system intervention to improve access, through ambulance service provision and improved quality of care at the district hospital. 18 The Western Area high referral index is likely multifactorial; the presence of tertiary hospitals offering higher levels of care; higher density of skilled healthcare workers 19 ; less distance and more availability of transport to access care; and a different socio-economic makeup to the national average. 17 Notably, national referrals to tertiary hospitals in the Western Area were very low 71 (0·5%) and do not account for the higher referral index observed. Speci c interventions to address disparities in referral numbers and service utilisation by district and to increase nationwide access and uptake of high-quality tertiary care should be considered. Our study did not demonstrate any signi cant relationship between bed occupancy or referral index with any of the SARA indicators reported, it may be that referral index is not a sensitive enough measure or may re ect limitations in the SARA methodology.
The high proportion of deaths in the Under 5 age category re ect national child health indicators 9 and district speci c reports. We did not nd a statistically signi cant association between time to arrive at facility and poor outcome, whereas moderate associations have been found by more appropriately designed case control studies in Sierra Leone. 20 The high odds ratios for mortality for adults not covered by the free healthcare scheme is likely multifactorial with adults only seeking care for severe disease due to cost barriers 10 and perceptions of poor quality care at Government hospitals. 21 The increasing OR for mortality as destination referral facility level increases suggests that a "triage and lter" role is being performed at the facility level with more serious and life-threatening cases being referred upwards through the health system. As expected, referrals seen within one hour by a clinician had a higher mortality, suggesting that patients are being triaged and seen according to severity at the facility level. The "triage and lter" theory is supported by a case-control study from Pujehun which found referral from primary healthcare as opposed to direct arrival at facility had an OR of 4.00 (1.98-8.43) for mortality, suggesting that less severe cases are being managed and ltered out at primary health care level. 20 In our study we found no signi cant difference in the national number of referrals, bed occupancy or outcome by month of the year, however analysis of multiple years of data will be better placed to detect seasonal variation at the national level. However, speci c hospitals especially the paediatric tertiary hospital in Freetown did experience signi cant seasonal variations in bed occupancy and number of referrals.
In addition to providing standardised data on patient referrals nationwide, we theorize that the NRS has bene ts and impact on multiple levels of the health system, from individual, to facility, to the national level. At the patient level, the RC network provides information on service and clinic timings, to ensure patients arrive in a timely manner, as well as support in arranging transport to access care. They provide information to the patient on service costs, a major barrier for non-free healthcare population, in addition to advocating for the free healthcare population to receive entitled free care. At the facility level, they are informed in advance of incoming referrals and mobilise the receiving clinicians and services (eg. blood bank, theatres) in order to reduce facility delays in care provision, this is critical in hospitals with limited human resource, where many services are dependent on one or two key clinical staff. In locations with inadequate blood bank storage, then this pre-warning is critical to mobilise appropriate donors. RCs perform daily bed occupancy rounds, to inform hospital decision makers and improve patient ow within the facility. Through supporting onward referral processes, once the decision has been made to refer a patient, they free up limited clinicians to concentrate on the next patient. At the system level, they provide national standardised data on referrals, informing health system planning. This highlights which lower level health facilities are over or under referring, allowing decision makers to focus system improvements. They stimulate a feedback loop, informing referring clinicians of the outcome of the referred patient, important for clinical learning, as well as supporting discharges back to the primary healthcare level for ongoing care. The network provides up to date information on signal functions, as well as service availability. Regular standardised bed occupancy reports demonstrate which facilities are overburdened and which hospitals are under-utilised. Finally, in the case of system failure, closure of hospitals or disruption to service availability, they are able to redirect referrals to functioning facilities.
The NRS dataset has been used to plan the deployment of ambulance services through the National Emergency Medical System (NEMS) which was launched towards the end of 2018. The NRS dataset will prove a valuable data source to evaluate the impact of the implementation of a nationwide ambulance system in Sierra Leone. Development and dissemination of the planned National Referral Form will increase standardisation of information collected on referrals across Sierra Leone. Further analysis of referrals against the standards laid out by the BPEHS may allow classi cation of appropriateness of referrals.
A recent systematic review of referral systems in LMICs concluded "from a methodological perspective there is no standard approach to assessing functionality and effectiveness of referral systems" 1 . It is therefore di cult to make value judgements on the dataset, for example, it is unclear whether the mortality rate per referral of 4.25% detected by our study is high or low. Referral systems are obviously intricately linked to the underlying health system, so developing international comparable standards is not feasible, although some have attempted to create national benchmarks 22 . Further attention should be paid to developing standardised de nitions, methods and indicators to assess quantity(referral index), appropriateness, timeliness and quality of referrals, this will allow comparison at the national or regional level. Monitoring of these indicators over time, or comparison of these indicators between districts or between facilities is a potential powerful method to understand health service utilisation and health system performance.
The wide variations in bed occupancy and referrals demonstrated by district highlight disparities in health service utilisation. The low rates of referral from district hospitals to tertiary care should be further investigated and barriers to tertiary care evaluated. We recommend longitudinal monitoring and inter district comparison of referral indices and referral mortality as a useful indicator of access and quality of care. Referral systems and pathways are critical to a well functioning health system and further research should be undertaken to standardise de nitions and identify the most appropriate indicators for analysis. Referral: A referral is the coordinated movement of a patient from one level or location of the health system to another level or location to provide appropriate care.
Referral Index = (number of annual incoming referrals/catchment population) x 1000 Incoming referrals are referrals into the receiving centre, Outgoing referrals are for onward referral of patients from the registering facility.
Incomplete referrals: Are referrals that are initiated but patient did not arrive at facility.
Counter referral is when a patient who has been received and assessed at the receiving facility is referred back to the sending facility.

Declarations
Ethics approval and consent to participate: Ethical approval was granted from Sierra Leone Ethics and Scienti c Review Committee.
Consent to publish: All authors reviewed the nal manuscript and approved of submission for publication Only fully anonymised data was used in the study.
Availability of data and material: All anonymised data generated or analysed during this study are included in this published article as an appendix.
Competing interests: The authors declare that they have no competing interests.
Funding: This research did not receive speci c funding. Figure 1 Nationwide referrals and mortality by month