Study setting
The research was conducted in Italy, within the 5-year SONAR project (Observational Study in Neonatology: Assistance and CaRe) set up in 1999 by the Italian Neonatal Network (INN). INN includes 93 NICUs with the goals of improving safety and quality of care in neonatal care by using research, training and quality improvement projects. Since 2004, INN has been part of the Vermont-Oxford Network, the most important neonatal network in the world, which includes more than 1,300 hospitals across 30 countries working to continuously improve neonatal care. 63 NICUs of the INN participated voluntarily in the SONAR project. SONAR’s aim was to identify the characteristics of NICUs, monitor outcomes and promote best practices [31].
Data collection
Our study includes 51 of the 63 NICUs that took part in the SONAR project. Operational units (OUs) that did not present complete data in managerial variables or outcomes were excluded from our sample. SONAR covered all healthcare professionals (doctors and nurses) who work in NICUs, preterm babies, and their parents. Questionnaires on the activities and managerial features of the OU were administered in written form to nurses, doctors and parents. The questionnaires differed for the three categories of respondent.
The questionnaires were developed by the Steering Committee of SONAR on the basis of a review of international literature and a meeting with health professionals. They were validated by the Advisory Board and the Steering Committee of SONAR, and pilot testing was conducted in several OU that took part in the project.
For this study we used only responses from medical and nursing staff.
The questionnaires were only administered to medical and nursing staff who had been working in the NICU for at least three months. Each NICU had a Local SONAR manager, belonging to the medical staff of the unit, who provided data about the activities and outcomes of its OU and was responsible for coordinating the project at local level.
Ethical approval
The study complies with the Italian regulations on observational clinical studies (Ministerial Circular n. 6 of 2/9/2002). The Ethics Committees of each centre involved in the SONAR project approved this study.
All the respondents to the questionnaire received an information sheet, containing a clear and comprehensive definition of the characteristics and objectives of the study. All the respondents gave written consent to the processing of personal data.
Managerial features
For each of the areas of the framework developed by Øvretveit for WHO [8], one variable was identified to describe the managerial features (MF) of the NICU. These features can be used to describe different managerial models.
MF1. Complexity of case
Local SONAR managers identified the acuity score for each neonate hospitalised in their own NICU. The acuity score is a tool developed by the Vermont-Oxford Network to define the type of patients admitted to the NICU, and includes five categories of increasing severity and care commitment [31, 32]. The lowest level of complexity indicates an infant who requires minimal care, while level 5 indicates an unstable infant who requires complex intensive care. Each NICU was thus assigned an acuity score given by the average of the acuity scores of all infants treated within it. NICUs that show a higher acuity score, nearer to 5, handle more complex cases on average and have a greater absorption of resources than NICUs with a lower acuity score nearer to 1.
In this study, we used the acuity score as an indicator available to managers (the head physician of the OU) for the evaluation of the complexity of NICU activities (MF1). It thus captures the first area of the managerial model: “Performance and results review”. Previous studies have used case complexity as a performance indicator [20, 33]. The United Kingdom Neonatal Staffing Study Group also suggests that case complexity, together with random chance and care quality, is one of the main elements leading to variability in outcomes in neonatal care [34].
MF2. Performance measurement
Doctors in their questionnaires were asked to indicate the system used in their NICU to evaluate qualitative performance by choosing one of the five following statements : 1) Quality measurement is sporadic; 2) The individual evaluates his or her own performance using information made available by the hospital; 3) Regular meetings between colleagues to discuss some clinical cases; 4) We jointly evaluate all cases not following defined procedures; 5) Periodical evaluation on quality indicators by external entities. The first and fifth statements express two extreme methods of evaluating performance. The first embodies a managerial approach poorly oriented towards quality assessment, which is left to the will of individual professionals. The fifth statement, on the other hand, presents a clear orientation of management to robust measurement systems. For each NICU, the score for the performance measurement system was defined through the evaluation expressed by its doctors. NICUs that show values nearer to 5 use more robust measurement systems than those with a score nearer to 1.
The use of performance measurement systems is the basis for activating benchmarking processes [35]. Managers can compare data collected from their measurement system with data from other hospitals, or with their own previously collected data, to develop adequate improvement processes. This study uses the type of performance measurement system (MF2) adopted to evaluate the “Benchmarking” area in the NICU management model.
MF3. Participation and support
The Practice Environment Scale of the Nursing Work Index (PES-NWI) is included in the questionnaire for nursing staff. PES-NWI is a tool developed by Lake for measuring the working environment and has a high degree of reliability and validity [36]. PES-NWI is made up of 31 items evaluated on a 4-point Likert-type scale, from 1 (maximum disagreement) to 4 (complete agreement). Items are grouped by Lake into five areas, two of which were used for this study. The “Nurse participation in hospital affairs” and “Nurse manager ability, leadership, and support of nurses” areas measure whether or not the hospital uses participatory leadership in managing activities. For each NICU, the average score was calculated in the two areas of the PES-NWI. An average close to 4 shows that an NICU is oriented towards participation, support and involvement of nurses. This element is used to identify the style of “Leadership” (MF3) in the NICU managerial model.
F4. Foundations for quality of care in nursing
Nurses are the professionals most involved in neonatal intensive care and are characterised by a very high level of skills. In PES-NWI, two items are useful for measuring NICU propensity to involve nurses more actively in care processes: 1) Use of nursing diagnoses; 2) Nursing care is based on a nursing, rather than a medical, model. These two items of the PES-NWI show to what extent the quality of care in the NICU is based on guidelines and procedures developed by nurses (F4). Together these two items can be used to measure the “clinical guidelines, protocols and procedure” area of the managerial model [37, 38]. As for the previous variable, as the value of the variable increases towards 4, the NICU propensity to use guidelines, protocols and procedures increases.
MF5. Motivation of medical and nursing staff
Doctors were asked to evaluate the level of satisfaction within their NICU on a four-point Likert-type scale (1: very low, 4: high), and the average level was calculated for each NICU. Motivation levels nearer to 4 reveal management ability to create a favourable working climate within the OU [39]. Managerial practices that promote job satisfaction and emotional support are also important for preventing burnout [40]. The motivation of medical and staff nursing (MF5) is thus closely linked to the last area of the managerial model: staff satisfaction.
Neonatal Outcomes
Neonatal outcomes used for this study were defined as for the Vermont-Oxford Network. Below is a brief description of each of the nine outcomes.
Mortality refers to intra-hospital mortality. Nosocomial infection indicates whether the infant has late bacterial infection (including Coagulase Negative Staphilococcus) after Day three of life. Severe Intraventricular Haemorrhage (severe IVH) indicates whether the infant has a Grade 3 or 4 periventricular-intraventricular haemorrhage. Cystic periventricular leukomalacia (PVL) refers to multiple small periventricular cysts identified on a cranial ultrasound, computerised tomography scan, or magnetic resonance imaging scan. Severe Retinopathy of Prematurity (ROP) indicates whether the infant has a stage 3, 4 or 5 ROP. Necrotizing Enterocolitis (NEC) was diagnosed at surgery, at post mortem examination, or clinically (bilious gastric aspirate or emesis, abdominal distension, occult or gross blood in stool) and radiographically (pneumatosis intestinalis, hepato-biliary gas, pneumoperitoneum). Pulmonary Bronchodysplasia (BPD) refers to oxygen at 36 weeks postmestrual age. Newborns discharged before 36 weeks do not have BPD. Morbidity indicates whether the infant survived with one of the following key morbidities: severe IVH, BPD, NEC, pneumothorax, nosocomial infection, or PVL. Human milk indicates whether the infant was discharged receiving human milk (including human milk fortifier and/or formula milk).
Outcome data were adjusted for confounders with covariance adjustment using linear regression (analysis of covariance, ANCOVA) [41], including, as for the Vermont-Oxford risk adjustment model, gestational age, gestational age squared, 1-minute Apgar score, small size for gestational age (lowest 10th percentile), multiple gestation, outborn status, gender, cesarean section delivery, and presence of a congenital anomaly [42]. For each outcome the adjusted mean was calculated for each statistical unit i.e. for each NICU, and the adjusted means were used for comparison between groups.
Data on outcomes were collected by the Local SONAR manager for each NICU. Low values associated with outcomes indicate better outcomes, except for the human milk variable, where higher values are preferable.
Data analysis
We conducted a cluster analysis to identify managerial models of NICUs. All variables MF1 to MF5 were included in the analysis to define the groups. Because outliers could significantly influence the results of the cluster analysis, variables were first standardised [43]. Patterns of respondents’ answers were then examined by applying the agglomerative hierarchical cluster analysis using the Ward method. The hierarchical cluster analysis with its dendrogram and statistics measuring the cluster fit (pseudo F-statistic) are useful to identify the number of the clusters. In a cluster analysis, in fact, the number of clusters is subjective. In our case, dendrogram and pseudo F-statistic suggested three clusters as the best option. Final clusters were thus formed using k-centered clustering with three groups. The method of Euclidean-distance was used in order to calculate differences between the variables. ANOVA was used to test whether the clusters significantly differ from each other in terms of the five managerial features used in the cluster analysis.
After forming the three managerial models, we conducted post-hoc tests (Bonferrroni) to assess differences between clusters on the neonatal outcomes.
Finally, in order to better describe the managerial models of NICUs, we also examined the association between other key elements of the NICU and cluster membership, which are useful for measuring the resources available to the NICU. The structural aspects were: size (number of beds), and human resource endowment (number of doctors and nurses per bed and per 1,000 day of length of stay). To investigate differences between clusters on structural aspects we used one-way ANOVA and Bonferroni’s test.
All statistical analyses were carried out using SPSS Statistics Version 25 and were conducted using a 5% significance level.
The NICUs of the sample
The 51 NICUs of our sample account for 81% of OUs belonging to INN, or 43% of all Italian NICUs. In Italy, NICUs can vary greatly in size and are sometimes very small. However, in terms of the number of preterm babies managed, the OUs belonging to INN deal with about 90% of all cases nationally, and the OUs in the SONAR project about 72%.
Twety-nine of NICUs in our sample are located in the north of Italy (56.9%), 6 in the centre (11.8%) and 16 in the south (31.4%). NICUs in the north account for 59.2% of the total population of existing NICUs in the north, those in the center account for 30.0% and those in the south account for 37.2%.
Overall, a total of 14,012 cases were treated in the sample NICUs during a period of 12 months (from March 2010 to February 2011), with an average of 274.75 per NICU (median: 245; Standard Deviation (SD): 145). Our study includes all infants who met the criteria of the Vermont Oxford Network Release 13.0 NOVEMBER, 2008, i.e. with a birth weight between 401 and 1500 grams or with a gestational age between 22+0 and 29+6 weeks. As for number of beds, NICUs have an average of 17.18 (median: 16; SD: 8.90) and range from a minimum of 4 to a maximum of 50 (SD: 8.90). Finally, there are on average 2.92 doctors and nurses per bed (SD: 0.65), and more precisely 0.76 doctors per bed (SD: 0.43) and 1.78 nurses per bed (SD: 1.08).
In sum, these data show that Italian NICUs work with a very different sets of resources and in widely differing organisational contexts.
For the 51 NICUs in the sample, 643 questionnaires were analysed from doctors and a total of 1,601 from nurses. The response rate of doctors was 96.8% and that of nurses was 96.6%.