Construction and psychometric validation of the Questionnaire on Patient Safety in the Operating Room

Background: Improving surgical safety is one of the objectives of the National Patient Safety Plan. Assessing the perception of nurses will contribute to a greater commitment to its application. Objectives: To construct and validate a questionnaire to evaluate patient safety in the operating room (OR). Methodology: Methodological research study. Development of a questionnaire, submitted to a Delphi panel, and study of its psychometric characteristics in a sample of 1,001 nurses. Construct validity study carried out by exploratory factorial analysis with Varimax rotation and convergent and discriminating validity. Its reliability was assessed using Cronbach’s alpha coefficient (α). Results: The validation by the Delphi panel resulted in a version composed of 79 items, which assess nine areas of patient safety (APS). Correlation matrices (p > 0.00) and Kaiser-Meyer-Olkin measures > 0.8 in the APS 9 ensure good adequacy of the factorial model, which groups the items into 19 dimensions with α values between 0.66 and 0.98. Conclusion: The questionnaire meets the validity requirements, revealing the high capacity for research use.


Introduction
The operating room (OR) is one of the most complex places of health care delivery and, consequently, where more adverse events (AE) occur, of which approximately 50% are preventable (Despacho n.º 1400-A/2015 de 10 de fevereiro, 2015).The recognition of this issue motivated the World Health Organization (WHO) to define surgical safety as a Second Global Patient Safety Challenge through the Safe Surgery Saves Lives project (SSSL; Organização Mundial de Saúde, 2009).In Portugal, the Directorate-General of Health (DGS) joined the program in 2009, which was considered a minimum quality standard (DGS, 2013).However, the National Plan for Patient Safety (NPPS) 2015-2020 mentions a low adherence to this project at the national level (Despacho nº 1400-A/2015 de 10 de fevereiro, 2015).This low adherence may affect patient safety (PS) substantially, thus making it essential to assess the implementation of the actions envisaged for the nine strategic objectives of the NPPS 2015-2020 in the ORs.Although quality and safety committees submit annual reports to the DGS describing the activities developed to meet the NPPS objectives, there is no instrument to allows assessing the degree of implementation of the NPPS actions from the perspective of healthcare professionals.The NPPS recognizes that the role of these professionals is to ensure compliance with the actions set out in the annual action plans of each hospital unit.As a result, assessing their perception is fundamental to diagnose the application of health policies within the context of PS.Due to the nature of provided care and because they make up the largest group of professionals in hospitals, nurses have a decisive impact on PS (Needleman et al., 2011).Studying their perceptions about the implementation of the NPPS will provide an operational overview of the greatest workforce in health institutions in highly complex settings like the OR.On the other hand, it is understood that this process will make nurses reflect on the topic and, therefore, further commit to a defined strategy for PS improvement.This study aims to construct and validate a questionnaire to assess PS in the OR and potentially to assess the implementation of the NPPS actions in the OR, from the perspective of nurses.

Background
Surgery became an integral part of health care worldwide, with an estimated 281 million surgeries performed per year.In industrialized countries, studies mention a surgical mortality rate of 0.4% to 0.8% and a higher complication rate between 3% and 17% (OMS, 2009).With an AE rate of 3% and a global mortality rate of 0.5%, surgery may be the cause of 7 million significant complications and 1 million deaths per year (OMS, 2009).Surgical care complications became one of the leading causes of death and disability worldwide, leading to surgical safety (SS) being regarded as a public health problem.
In the Netherlands, a study reviewed 7,926 clinical processes of 21 hospitals and concluded that the AE occurring in the OR represent approximately 65% of all AE in all healthcare settings.Also, they are more preventable (41%) and have more severe consequences (Zegers et al., 2011).Incidents occurring in the OR are diverse and are often related to infections, hemorrhages, injuries caused by mechanical, chemical, or physical factors, communication, equipment, and administration of medication and blood products (Heideveld-Chevalking, Calsbeek, Damen, Gooszen, & Wolff, 2014;Moura & Mendes, 2012;Zegers et al., 2011).When discussing their causes, studies indicate that human factors form the basis of most surgical incidents (Heideveld-Chevalking et al., 2014;Thiels et al., 2015;Zegers et al., 2011). Heideveld-Chevalking et al. (2014) concluded that the majority of incidents is associated with non-compliance with standard clinical guidelines (27.5%), 15.4% with human errors or omission, and 11.5% with communication problems.In this respect, the WHO points out as one of the four main problems about PS the lack of systematization in the use of available evidence (OMS, 2009).Therefore, the Surgical Safety Checklist (SSC) was developed and integrated into the SSSL Project, which revealed to have a very significant impact on PS, leading to the decrease of approximately one-third of the number of deaths and surgical complications (Haynes et al., 2009).However, the implementation of the project has encountered some degree of resistance at the international level (Russ et al., 2015) and the national level (Despacho nº 1400-A/2015 de 10 de fevereiro, 2015).In the face of this issue, it is imperative to combine efforts for the application of the policies of PS in the OR.In Portugal, the health policy about PS is defined by the Despacho n.º 1400-A/2015 de 10 de fevereiro (2015), which approves the NPPS 2015-2020, integrated into the 2015-2020 Health Quality National Strategy (HQNS).The NPPS is part of a public policy to fight against the factors that contribute to security incidents associated with healthcare delivery.This paper aims to support health managers and healthcare providers in adopting strategies to improve the management of inherent risks of healthcare delivery.Therefore, the NPPS includes nine strategic objectives, as well as actions to be developed and targets to be achieved by all healthcare organizations of the National Health Service (NHS).The implementation of the NPPS requires the involvement of all members of governance and coordination and the operational practice of health care delivery.Healthcare professionals should ensure compliance with the actions of each organization's plans.Hence it becomes crucial to involve professionals in the assessment of the level of implementation of the NPPS actions.The construction and validation of a questionnaire with the potential to evaluate, from the perspective of nurses, the PS, namely the implementation of the actions of the NPPS in OR, it becomes essential to perform an operational diagnostics of the achievement of strategic objectives.This diagnostic evaluation will raise awareness among professionals involved in direct provision of care, including nurses, to the strategy defined in the context of PS.Contribute to generate greater commitment of nurses in their implementation and analyze their strengths and weaknesses, potentiating the definition of intervention strategies with a view to the continuous improvement of processes, thus helping to promote the PS in OR.

Methodology
To meet the delineated objectives, a quantitative study was carried out in two phases: preparation of a questionnaire to assess the perception of nurses on the PS in the OR, particularly, on the level of implementation of the actions provided for in the NPPS, and study of its psychometric properties.For the formulation of the questionnaire, the methodological framework for the formulation of instruments proposed by Pasquali (1999) and Moreira (2009) was used.The initial version was submitted for analysis by a panel of experts, and its psychometric properties were analyzed, leading to the validation of its final version.

Preparation of the initial version
The initial version was based on the actions defined in the NPPS 2015-2020 (Despacho n.º 1400-A/2015 de 10 de fevereiro, 2015).The WHO and DGS rules and guidelines regarding PS and healthcare quality constituted important theoretical support for its formulation (Despacho n.º 5613/2015 de 27 de maio, 2015;DGS, 2013;DGS, 2015;DGS, 2017;OMS, 2009).Actions of the NPPS were considered, whose implementation is the responsibility, according to the NPPS, of the health care institutions.Actions not related to hospital activity and the OR were excluded.The initial version of the questionnaire was composed of 72 items, organized in nine areas for patient safety (APS), which correspond to the nine objectives of the NPPS 2015-2020.

Semantic and content validation
For semantic and content validation, the initial version of the questionnaire was submitted for the assessment of a Delphi panel composed of seven specialists OR nurse, who have occupied leadership positions in the Portuguese Operating Room Nurses Association (Associação de Enfermeiros de Sala de Operações Portugueses -AESOP) and possess professional experience in different ORs.For the constitution of the group of experts, the criteria were professional activity in the OR of at least 10 years, the title of nurse specialist, master's degree, and work production within the scope of PS in the OR (projects, publications, participation in working groups).The responses of the participants were sent to the e-mail address of the researcher, created exclusively for this purpose.Participants were asked to express their agreement/disagreement with the items of the questionnaire, on a scale of 1 (I completely disagree) to 5 (I completely agree).They were given the possibility to suggest possible changes, both in the semantic construction and the addition of new items.A level of agreement above 75% was defined as a criterion of acceptance (Menino, Dixe, & Louro, 2016).Thus, all the changes were accepted and, after three rounds, were agreed upon by at least six participants.In the first round, the participants in the Delphi panel suggested the addition of three new items related to the standardized clinical record process (APS 2 "Communication safety"), to the conduction of the SSC checks in the moments defined by the WHO/DGS, and to the participation of all the elements in the implementation of the SSC (APS 3 "Surgical safety").All other items fulfilled the criterion of acceptance.Subsequently, there was a new round after the suggested items were formulated.In the second round, the items generally complied with the criteria of acceptance, but the addition of four new items was suggested, related to availability in the services of devices for the prevention of falls and pressure ulcers (PU) that are suitable and sufficient in number (APS 6 "Prevention of falls" -2 items; APS 7 "PU prevention" -2 items).These items were formulated, and a third round was held, in which the 79 items obtained a level of agreement within the criteria of acceptance.After the completion of the three rounds, the questionnaire was submitted to a pre-test applied to 10 nurses not included in the study.There were no suggestions and difficulties manifested.This process resulted in a version composed of 79 items, with a five-point Likert-type scale from 1 (never) to 5 (always), theoretically grouped in 9 APS as follows: (1) APS1 "Internal environment safety culture" (items b1; b2; b3; b4; b5); (2) APS2 "Communication safety" (items c1; c2; c3; c4; c5; c6; c7; c8); (3) APS3 "Surgical safety" (items e1; e2; e3; e4; e5; e6; e7; e8); (4) APS4 "Medication use safety" ( f1; f2; f3; f4; f5; f6; f8; f9; f10); (5) APS5 "Unambiguous patient identification" (g1, g2, g3, g4, g5, g6, g7, g8, g9); (6) APS6 "Prevention of falls" (items h1, h2, h3, h4, h5, h6, h7, h8, h9); (7) ASD7 "PU prevention" (items i1, i2, i3, i4, i5, i6, i7, i8, i9; i10; i11); (8) ASD8 "Notification, analysis, and prevention of incidents" (items j1, j2, j3, j4, j5, j6, j7, j8, j9, j10); (9) ASD9 "Antimicrobial resistance and infection control and prevention " (k1, k2, k3, k4, k5, k6, k7, k8, k9).Thus, a version of the questionnaire was considered to meet the conditions for continuing the study of its psychometric characteristics.

Population/sample under study
The target population of this study is composed of nurses who work in the OR (of adult patients) of 24 NHS hospitals, meaning a total of 46 ORs.The criterion for inclusion in the sample was to have professional activity for over 6 months.The criteria for exclusion were: a position as head nurse and temporary absence from work during the period of data collection.Data collection took place between January and October of 2018.A total of 1,798 questionnaires were applied to all nurses of the 46 ORs who met the inclusion and exclusion criteria.One thousand and one properly completed questionnaires were returned, which represents an adherence rate of 55.70%.

Ethical-legal aspects
A position statement was requested to the Ethics Committee of the Research Unit in Health Sciences: Nursing of the Nursing School of Coimbra to comply the ethical-legal rules.Also, a request for formal authorization of information collection was delivered to the Administration Boards (AB) from the hospitals involved in this study.The abovementioned Ethics Committee and the AB of the 24 hospitals provided a positive position (P 458-09-2017; P 464-10-2017).Afterward, a meeting was scheduled with the head nurses of the different ORs, to explain the objective of the study and request their collaboration in the distribution of questionnaires.The questionnaires were delivered in envelopes opened, along with the informed consent.The participants were informed to deposit them inside the ballot box, once they completed the questionnaires.The informed consent was collected separately.

Data treatment
For the data treatment, the IBM SPSS Statistics software, version 25.0, was used.The data analysis used descriptive and factorial statistics.In the descriptive analysis, both frequencies (absolute and percentage), measures of central tendency (mean, maximum, and minimum), and measures of dispersion (standard deviation) were calculated.Exploratory factor analysis was applied to each of the nine APS to assess the validity of the instrument.For factorial extraction, the principal components analysis was preferred to the common factor analysis because the purpose was the reduction of data, the lowest complexity, and the clearest interpretation.The eigenvalue criterion was respected, retaining only factors with eigenvalues ≥ 1.To facilitate the interpretation of factors, thus minimizing the number of variables with high factor loadings, orthogonal Varimax rotation was used.For item retention, the items' factorial loading ≥ 0.45 was relevant because it explains at least 25% of the variance.Pearson's correlation test was used to assess the convergent and discriminant validity (correlation of each item with their dimensions).The reliability assessment was calculated through the internal consistency of each dimension using Cronbach's alpha coefficient (α).

Sample
The sample consists of 1,001 nurses, mostly female (84.90%).The respondents have a mean age of 42.74 years (SD = 0.27) and have, on average, 19.76 years (SD = 0.27) of service.These nurses work in the OR and their current service for 13.52 years (SD = 0.28) and 11.56 years (SD = 0.27), respectively.Regarding their academic qualifications, the majority of the professionals have a licenciatura (79.10%), 18.50% have a master's, 1.90% a bacharelato, and 0.50% a Ph.D.Only 17.90% are specialited nurse.The majority of respondents work in central ORs (76.9%), 15.60% work in outpatient ORs, and 7.40% in peripheral ORs.More than half of the nurses work in accredited/certified ORs (59.7%).

Validity and reliability of the scales
The sample adequacy tests used were Keiser-Meyer-Olkin test and Bartlett's test of sphericity.The first test presented values between 0.82 and 0.88, and the result of the second test was χ2 ≥ 3572.46 and ≤ 10180.79(p < 0.00), in the nine APS, meaning there all the necessary conditions are present to continue the factorial analysis of each one of the APS.The Kaiser criterion suggests that two factors should be extracted in the majority of the APS (APS2, APS3, APS5, APS6, APS8, APS9), with the exception of APS4 and APS7, in which KMO suggests the extraction of three factors, and APS1, the extraction of one factor (Table 1).The extraction of factors with eigenvalues above 1 explains its variance between 58.58% (APS9) and 80.02% (APS7).The orthogonal Varimax rotation was used to facilitate the interpretation of factors, by polarizing them.All items presented saturation levels in the factor ≥ 0.48 (f1), which demonstrates a clear relation between the variables and the extracted factors, with the exception of items k2 "Trichotomy is avoided when preparing the surgical site" (0.31; APS9) and j6 "When performing a notification, I use the NOTIFICA notification system" (0.33) of the APS8, whose values would recommend their removal (Table 1).However, it was decided to keep the items at this stage because of their theoretical relevance, and this decision was validated after the analysis of each dimension's internal consistency and the item's correlation with its dimension.Subsequently, meanings were assigned to the dimensions according to the interpretation of factor loadings for the variables, meaning that the highest loadings influenced more the name assigned to the dimension.The majority of dimensions are related to the implementation of good practices (D2; D4; D6; D9; D11; D13; and D18) and audits (D3; D5; D7; D10; D12; D14).In addition, dimensions related to the internal environment safety culture (D1), prescription practices (D8), availability of resources for PU prevention (D15), incident notification (D16), incident analysis and prevention (D17), and epidemiological training and monitoring on antimicrobial resistance and infection control and prevention (ARICP; D19) emerged (Table 1).The study of reliability included assessing the internal consistency of the dimensions obtained after factor analysis by calculating the Cronbach's alpha coefficient (Table 1).The dimensions presented α values between 0.73 (D9) and 0.98 (D12 and D14), evidencing that the analysis by dimension has a reasonable/very good internal consistency, except dimensions D2 (0.69) and D18 (0.66; Table 1; Pestana & Gageiro, 2014).The values of corrected correlation of each item with its dimension ranged between 0.41 (k3) and 0.96 (h7 and h9), except items j6 and k2, which presented 0.26 and 0.27 values, respectively.Based on the correlation values > 0.20 and the theoretical relevance of these items to the scale, the decision to maintain them in this phase was taken and will be considered after the analysis of the convergent validity and discriminant validity.All items presented convergent validity and discriminant validity.Indeed, all items revealed a higher correlation with the total dimension to which they belong than with the dimension to which they do not belong, even in the case of items with low saturation in the factor (j6 and k2).In the APS 8 dimensions (D16, D17), it was noted that the correlation of item j6 with the dimension to which it belongs (D17), although low (r = 0.26; p = 0.00), is higher than the correlation with dimension D16 (r = 0.22; p = 0.00).Furthermore, in the APS 9 dimensions (D18 and D19), the correlation of item k2 with the dimension to which it belongs (D18; r = 0.64; p = 0.00), is much higher than the correlation with the dimension it does not belong to (D19; r = 0.28; p = 0.00).In the face of these analyses, it was decided to maintain these items.
In a nutshell, the final version of the Patient Sa-fety in Operating Rooms questionnaire (PSOR) was composed of 79 items, grouped into 19 dimensions, which evaluate nine APS corresponding to the nine NPPS strategic objectives.

Descriptive results of the PSOR questionnaire
The descriptive analysis of the dimensions allows emphasizing that all dimensions obtained values ≥ 2.50 (M), except dimension D14, which showed lower values in the middle point (M = 2.35; SD = 0.04; Table 2).The dimensions that evaluate audits reveal a low level of implementation of the actions provided for in the NPPS (mean values below 3), except audits regarding the SSC (D7).On the other hand, dimensions D9 (M = 4.69, SD = 0.02) and D18 (M = 4.06, SD = 0.02), present, from the perspective of participants, more robust levels of implementation (Table 2).

Discussion
This study had the participation of around onethird of the Portuguese perioperative nurses, taking into account the results of the Assessment of the National Situation of Operating Rooms, which mentions the existence of 2,975 perioperative nurses in Portugal (Ministério da Saúde, 2015).The sample size is significantly higher than the recommended by Pestana and Gageiro (2014), presenting a ratio of valid responses per variable of 12.67, meaning more robust results.
The questionnaire SDBO presents good validity and reliability indicators, as evidenced by the results of the exploratory factor analysis, principal components, convergent and discriminant validity, and α values.Although the internal consistency values in two dimensions (D2 and D18) were low (0.69 and 0.66, respectively), the authors believe that they meet the acceptable minimum namely because all items show the correlation of the item with the dimension above 0.20 and total α values of the dimension > 0.60.In this sense, although some caution in their interpretation is recommended, they were maintained.
Regarding the assignment of meanings to different dimensions, it stands out that the items related to the implementation of good practices and the audits were grouped into different dimensions, allowing to carry out a differentiated analysis on the implementation of best practices within the context of PS and its monitoring by the audit process.
From the perspective of nurses, good practices in the unambiguous patient identification (D9) and antimicrobial resistance and infection control and prevention (ARICP; D18) present more robust levels of implementation possibly because issues about the unambiguous patient identification are associated with sentinel events involving serious consequences for patients, health professionals, and organizations (DGS, 2013).The positive perception of the implementation of the good practices of ARICP may be because surgical site infections (SSI) constitute one of the main complications associated with surgical procedures, meaning that professionals should be more attentive to this issue and focused on the implementation of measures to prevent this scourge.The questionnaire can be applied in its entirety or a piecemeal approach, according to the APS intended for analysis, considering that the psychometric study allowed achieving reliable dimensional constructs.
Because of its capacity for diagnosing the implementation of health policies within the context of the OR, as a part of the HQNS and the NPPS, it is considered that the annual implementation of the PSOR questionnaire could constitute an important tool so that the services and Safety and Quality Committees of hospital organizations can define actions of continuous improvement and development of the safety culture.
The authors consider as a limitation of this study the fact that only the assessment of the perception of nurses was assessed.Taking into consideration that PS is a collective responsibility and a concern for all health contexts, it is important that future studies take into account the perceptions of other professionals and focus on other contexts of health care delivery.The authors believe that this questionnaire can be easily adapted to other contexts, by simply deleting the items about the OR.

Conclusion
This study allowed the construction and validation of a questionnaire that assesses the perceptions of nurses on PS in the OR, particularly through the evaluation of the level of implementation of the NPPS actions in the OR.The final version was composed of 79 items, organized into 19 dimensions, which allow evaluating nine APS, corresponding to the strategic objectives of the NPPS.The PSOR questionnaire presents a good level of validity and reliability, allowing to conclude that it can assess the perception of professionals who work in this setting.It is considered that this questionnaire is an important tool to raise awareness of professional healthcare providers at the operational level, leading to greater involvement of these professionals in the diagnosis of the situation and contributing to greater commitment to the implementation of safety policies and quality improvement.Although, in this phase of the study, the researcher deliberately chose to focus only on the perception of nurses of the OR, it is believed that a better understanding of the reality would require the study include the participation of other professionals involved in the implementation of the actions provided for in the NPPS, namely anesthesiologists and surgeons and other health care professionals.Thus, it would be appropriate to apply them to abovementioned professional groups and to adapt the questionnaire to other contexts.

Table 1
Matrix of factorial analysis with varimax rotation of the items of the nine APS, statistics of homogeneity of the variables and Cronbach's internal consistency coefficient of the nine APS(n = 1,001) ANA SOFIA DE CARVALHO MOTA et al.Construction and psychometric validation of the Questionnaire on Patient Safety in the Operating Room Note.APS = Areas of Patient Safety.

Table 2
Descriptive analysis of the dimensions of the PSOR questionnaire: Cronbach's alpha, minimum,  maximum, mean, and standard deviation (n = 1,001) Note.N = Sample; Min = Minimum; Max = Maximum; M = Mean; SD = Standard deviation.