Acceptability of Australian prehospital care quality indicators: An explanatory sequential mixed methods study

Background Systematically developed quality indicators (QIs) facilitate the measurement and monitoring of quality of care and ultimately meaningful quality improvement. The aim of this study was to evaluate the acceptability of a predetermined suite of 84 scientifically valid prehospital care QIs from the provider perspective. Methods An explanatory sequential mixed methods study design was used. Quantitative data were obtained from 36 participants of an online survey in which they rated the acceptability of the QIs using a five-point numerical rating scale. Qualitative data were gathered by conducting semi-structured interviews with a purposive sample of nine survey participants. The successional collection of quantitative and qualitative data facilitated integrated interpretations and conclusions about the acceptability of the QIs. Results Generally, the acceptability of all QIs in the suite was rated highly. Data suggested a positive association between acceptability and other key characteristics of QIs. QIs that were seen as clear, supported by scientific evidence, practical and meaningful tended to be more acceptable than those which were not. The benefits of outcome type QIs were recognised, but participants raised concerns about their sensitivity in the measurement of prehospital quality of care. To be acceptable, QIs which included time intervals needed to be specific about time-critical interventions. Further, the high acceptability of the QIs was explained by a connection to participants’ professional values and qualities. Assessing the QIs’ acceptability from a healthcare provider perspective meant that QIs on patient satisfaction frequently received lower ratings. Conclusion The findings of this study provide evidence of the acceptability of prehospital care providers of a proposed suite of QIs. Future research should evaluate the feasibility and reliability of the QIs. There is also a need to investigate how acceptable the proposed QIs are to patients and communities.

itself. [2][3][4] This has advanced the scientific rigour around the approaches and methods used for selecting and developing single QIs addressing specific aspects of quality, and QI suites addressing broader areas such as prehospital care. [5][6][7] Whilst scientific validity is a minimum prerequisite for any QI, subsequent developmental work should aim to provide empirical evidence, as far as possible, of a number of other key characteristics, including acceptability. 5,6,8,9 Acceptability is a multi-faceted construct, 10 and in the development and application of QIs depends on the extent to which measurement of performance based on a particular QI is acceptable to both those being assessed and those undertaking the assessment. 8 Quality measurement is not synonymous with quality improvement. However, positive change cannot occur without meaningful measurement of performance. 2 For measurement to be effective in facilitating improvement the gathered intelligence needs to be able to influence decision-makers. If decision-makers and key stakeholders do not accept a QI, the results of associated measurement will not be useful for influencing people to make a change. 5 Therefore, the potential of a QI to facilitate quality improvement relies on it being acceptable to stakeholders.
An all-inclusive definition of prehospital care comprises all healthcare services and resources prior to referral to a hospital, if needed. However, for the purpose of this project, prehospital care was confined to the care and services provided by ambulance services. In Australia, the provision of prehospital care is performed predominantly by jurisdictional state/territory ambulance services forming an important part of the national health system. In 2020/2021, these ambulance services had an operational workforce consisting of 16,742 staff and responded to over 4 million incidents (158.7 per 1,000 population). 11 Ambulance services in Australia, like most other healthcare services anywhere, are under pressure to maintain contemporary, high-quality patient care in an environment with constantly growing demands and complexity. 12,13 The right measurement of the right data over time, and its use as performance intelligence, plays a pivotal role in guiding any healthcare services' decision-makers with respect to quality of care. 14 Whilst a plethora of healthcare QIs exist, 15 relatively few have been developed specifically for prehospital care. 16 Moreover, of these prehospital care QIs many have a focus on operational aspects (such as response times) and have been developed using non-systematic methods. 16 Notwithstanding more sophisticated servicespecific QIs developed by Australian state/territory ambulance services, the quality of care delivered to Australian residents across its jurisdictions is measured in accordance with a national performance indicator framework. 11 The measures therein address response times, patient satisfaction, workforce and a small number of clinical conditions, including cardiac arrest and pain management. These measures do not reflect the breadth and complexity of care and services that modern ambulance services provide. As such, there is a need for a contemporary and meaningful suite of prehospital care QIs that Australian ambulance services could draw from for national reporting, benchmarking and quality improvement purposes.
This study formed part of a larger research project aimed at developing and testing prehospital care QIs for the Australian setting (www.aspireproject.net). 17 An initial scoping review was conducted in accordance with JBI methodology to locate, examine and describe the international literature on indicators used to measure prehospital care quality. 16 Subsequently, a modified RAND/UCLA appropriateness methods (RAM) was undertaken to aggregate the QIs identified in the scoping review, develop a suite of prehospital care QIs, and to assess the QIs for clarity and validity. 18 Preparatory work for the expert consensus process included streamlined evidence syntheses guided by the JBI approach for rapid reviews and evidence summaries. 19,20 The current study set out with the aim to gain insight into the acceptability of a predetermined suite of 84 scientifically valid prehospital care QIs from the perspective of paramedics and ambulance service managers.

Methods
The methods applied in this study and other parts of the project were specified in advance in a protocol. 17 Data collection for this study was commenced in February 2021 and completed in August 2021. The authors have no conflict of interest to declare, and the research was conducted independently and not at the behest of any regulatory body or ambulance service.

Preceding work
The 84 proposed QIs stemmed from previous published studies of the project, namely an initial scoping review, 16 and a subsequent evidence-informed expert consensus process. 18 In preparation for the consensus process, QIs identified in the scoping review were aggregated and systematically prepared within clinical and non-clinical domains, and a structure/process/outcome and access/safety/effectiveness taxonomy as summarised in Tables 1 and 2

Study design and setting
A two-staged explanatory sequential mixed methods research design was adopted. 23,24 Inquiries within the two stages of the study were guided by appropriate research paradigms depending on the specific objective of each stage. A postpositivist stance was taken in the initial quantitative stage followed by a constructivist stance in the subsequent qualitative stage. 23 Thus, researchers focused on the research questions and incorporated plurality of methods to answer them. 25 Using Crotty's conceptualisation, 26 the study was further positioned within social science theories informed by reviews and frameworks of acceptability as a criterion for evaluating performance measures. 10,27,28 In Stage 1, an online survey was conducted to collect quantitative data on the acceptability of the 84 QIs. In Stage 2, online one-to-one semi-structured interviews were performed aimed at qualitatively explaining what makes QIs acceptable or unacceptable. Although results of the quantitative and qualitative aspects were integrated, the qualitative stage constituted the core of the research. Integration occurred at the conceptualisation of the study by planning an explanatory sequential design. During the research, integration was achieved through linking data collection and analysis. This was done by connecting through sampling, building by considering the results of the survey during the interviews, and merging the two datasets for analysis. 23,29 Integration through narrative was applied using a contiguous approach in reporting the results of the two stages, followed by weaving in the discussion. 29,30 The study was conducted in Australia.

Participants and recruitment
The target population for this study was comprised of paramedics and directors, managers, or supervisors who have worked on quality improvement projects from any Australian ambulance service. Recruitment involved website and email advertisement by the Australasian College of Paramedicine (ACP), the peak professional organisation with a membership of around 8,000 paramedics and paramedic students, 31 followed by numerous social media posts on Twitter, LinkedIn and Facebook over a four-week period. At the end of the survey, participants of Stage 1 could express their interest in participating in Stage 2. The sample for Stage 2 was purposively selected and aimed at an even representation of

Stage 1 data collection
Participants were asked to anonymously complete an online survey (designed on Qualtrics; Qualtrics, Provo, Utah, USA). Since there was no existing survey that met the needs, the survey was purpose-built. The survey collected basic demographic data and then asked participants to answer the following question for each QI using a fivepoint numerical rating scale (1 = very unacceptable, 2 = unacceptable, 3 = neutral, 4 = acceptable, 5 = very acceptable): How acceptable is it to assess the quality of your patient care or the quality of your ambulance service based on data collected using this quality indicator? 'Neutral' signalled ambivalence in the participant's consideration of the acceptability or unacceptability of a QI. Acceptability was defined as the degree to which something is tolerated, allowed, or okay to use. Considering the brevity and simple survey flow as well as the earlier work optimising clarity of the QIs, piloting was not deemed necessary. Survey settings prevented multiple submissions by individuals. Based on the Australian registered paramedic population of approximately 17,000, using a sample size estimation with a confidence level of 95% and, for practical reasons, accepting a margin of error of 8%, an ideal sample size of 149 was pursued.

Stage 1 data analysis
Quantitative data analysis was performed using Microsoft Excel V16 (Microsoft, Richmond, Washington, USA) and IBM SPSS Statistics V27 (IBM, Armonk, New York, USA). Descriptive statistics were completed to summarise all survey items. For each QI, central tendency of acceptability ratings was evaluated using the median. The fivepoint rating scale was assumed to represent a continuous variable rather than five discrete categories, and medians were calculated accordingly. 32,33 Diverging stacked bar charts were created to visualise distributions. Explicit acceptability and unacceptability were calculated to be expressed as percentages by combining ratings of four (acceptable) and five (very acceptable), and two (unacceptable) and one (very unacceptable), respectively. Kruskal-Wallis tests (KWt) were conducted to examine the differences on medians according to QI types (structure, process and outcome) and quality dimensions (access, safety and effectiveness). A p value of <0.05 was considered statistically significant. Finally, the median of medians (as the preferred measure of centre in light of the distribution of medians) was identified for the entire suite, its two domains, as well as subsets of QIs in accordance with the project's classification system.

Stage 2 data collection
Guided by methods described by DiCicco-Bloom and Crabtree, 34 an interview guide containing a priori questions and topics/questions informed by the results of Stage 1 was developed. The interview guide facilitated a flexible interview protocol and was supplemented by follow-up questions, probes and comments. [35][36][37] The openended questions aimed at eliciting explanations for what makes QIs acceptable or unacceptable to participants.
Participants were asked specifically about QIs which were rated relatively low or high in Stage 1. To ensure diversity and to optimise credibility of results, maximum variation sampling was used from the pool of volunteering participants. 38 Interviews, with an intended length of approximately 30 min, were conducted in English by the principal investigator (RP) using audio call on Zoom (Zoom Video Communications Inc., San Jose, CA, USA) and recorded for transcription. During the interviews, member checking was performed through the interviewer restating or summarising information and then questioning the participant to determine validity. 35 Field notes were taken during and after each interview. Data were collected until saturation, defined as the point at which no new information was observed. 39,40 One additional interview was conducted for assurance confirming no new themes were observed in the data. 41 Trustworthiness of the findings was enhanced by data triangulation of recorded interviews, transcripts and field notes. 23

Stage 2 data analysis
Using NVivo 12 (QRS International, Doncaster, Australia), the transcripts were analysed by conducting thematic analysis as outlined by Braun and Clarke. 42 Analysis was performed by one researcher (RP) using an inductive and semantic approach. The triangulation which was performed to ensure credibility also enabled content familiarisation (phase 1). Data were disassembled through coding (phase 2) by systematically identifying aspects of interest and collating those relevant to each other. Codes were then reassembled by placing them into context with each other to create potential themes and, where appropriate sub-themes improving structure to relatively large and more complex themes (phase 3). Themes were reviewed developing a thematic 'map' of the analysis within NVivo (phase 4). Ongoing analysis facilitated the refinement of the themes and eventually the allocations of definitions and names for each theme and sub-theme (phase 5). Finally, compelling extracts were selected to perform conclusive analysis and relate the analysis to the research question on acceptability (phase 6). 42 During each of the six phases, means of establishing trustworthiness as described by Nowell and colleagues 43 were applied.

Stage 1
Of 44 responses to the survey, 36 (82%) were complete and thus utilisable for data analysis. Participant demographics are detailed in Table 3. Participants were predominantly male (61.1%) and relatively young (72.2% aged between 25 and 44 years). Half of all participants had a paramedic specialist qualification, and more than half (58.3%) were from New South Wales (NSW). Experience ranged from less than five years to more than 24 years. Two-thirds of participants worked primarily as clinicians, whereas the other third chiefly had managerial responsibilities.
Overall, the acceptability of all QIs in the suite was rated highly. Figure 1 shows the left-skewed distribution of acceptability rating medians for all QIs and further results for all QIs of the suite can be found in Supplementary Appendix A. Table 4 highlights those QIs which received relatively low acceptability ratings. Kruskal-Wallis tests showed that there was a statistically significant difference in medians between the different QI types (structural, process and outcome) (KWt(2) = 13.260; p = 0.001), however, no significant difference in medians between QIs addressing the three quality dimensions (access, safety and effectiveness) was found (KWt(2) = 0.437; p = 0.8).
The median of medians for all QIs in the suite was 4.60 (IQR 0.32). The medians of medians for QIs within the organisational/system and the clinical domains were identical and equal to that of all QIs (4.60; IQR 0.33). Medians of medians for structural, process and outcome indicators

Stage 2
Data saturation was achieved after eight interviews. The additional ninth interview yielded no new information. Six of the nine interviewees were clinicians and four were managers/supervisors. Participant demographics are summarised in Table 3 and individual attributes can be found in Supplementary Appendix B. The mean interview duration was 29 min, ranging from 15 to 45 min. Overall, five themes and six sub-themes were created (Table 5).
Theme 1: Characteristics of QIs. To increase the level of acceptability, a QI needed to possess certain characteristics which made it suitable to be implemented for its intended use (i.e., the measurement of quality). These characteristics included clarity, being evidence-based, practicality and meaningfulness. The more a QI was perceived to have these desirable characteristics, the more acceptable it was to participants. Vice versa, QIs which lacked these attributes, were less acceptable. This participating manager explained why they rated the acceptability of QI-B.1.3. relatively low. The phrases "unless specifically indicated" was considered vague and insufficient. For this QI to be more acceptable, it needed to detail certain patient variables to describe the exact clinical scenario when the treatment of interest should be administered.
Sub-theme 1b: Evidence-based. When asked what makes a QI acceptable, a common response from participants was  that they should be based on best available evidence. Such QIs were more valid and thus more acceptable to participants. On the other hand, some QIs which were seen as being ill-supported by evidence was considered less acceptable.
[QIs] should be evidence informed. There's no point having quality indicators that say, "do X", if×is way out of date. (Participant #3) I would say that having an evidence base to support methodologies is pretty important.
Because what you want to use is validated approaches that give some credibility to whatever you report. (Participant #8) Sub-theme 1c: Practicality. Practicality influenced the level of acceptability. Especially participating clinicians considered those QIs which in their eyes described realistically achievable and contextually viable aspects of prehospital care to be more acceptable. Often this was described in terms of the holistic service delivery rather than sub-sets of patient encounters. Thus, there was an element of equity, too.
Quality indicators need to ensure that quality healthcare is delivered, but that it's attainable by all members of the workforce. (Participant #1) One participant talked about a major trauma patient they recently treated and how QI-B.6.4. would not have been met and how this QI details an unrealistic practice.
But all of this takes way more than 10 min. It's totally unrealistic to be suggesting that we're going to be on the road on the way to definitive care in 10 min. It's just not happening. You know what I mean? I don't even see how it can. (Participant #4) Some participants identified a potential risk of 'pleasing the QI' when impractical QIs are implemented, meaning that there is a conflict between what the QI describes and what is in the best interest of the patient. Paramedics may reluctantly provide patient care that is in line with the QI but would see true quality of care being compromised. This made participants comment on flexibility in the interpretation of measurement data and the associated differentiation between warranted and unwarranted variation. Variation from what a QI dictates was considered to be warranted when it is in the best interest of patients' clinical needs or preferences. Sub-theme 1d: Meaningfulness. An acceptable QI described aspects of care that were meaningful to participants. In other words, they needed to describe aspects of prehospital care that conform to the individual participant's ideology of quality in this context. Of the many dimensions of quality, participants placed most emphasis on effectiveness. QIs that focus on the impact of prehospital care provided to patients and communities were considered more acceptable.
Safety and patient-centredness also featured as desirable attributes of quality.
It has to be something that's generally involved in patient outcomes as best as possible, and that's multi-faceted. So that's not just response time to getting there, but it's actually what we're able to do for the patient once we are there. (Participant #2) One of the managers commented on the purpose of measurement often being lost in large organisations and questioning the meaningfulness of the collected and analysed data.
Sometimes I think when you get into big organizations […] we ask big questions, and we ask for big packets of data. But we're not really sure what we're asking, why we're asking for it, and what its application is in terms of the strategic information and overview and what is it going to drive us to change or verify. (Participant #8) Theme 2: Patient satisfaction. QIs that describe aspects of patient experience and satisfaction was seen by the participants as less important and limited in their validity to be used as measures of quality.
Sub-theme 2a: Lesser priority. The measurement of patient experience and satisfaction was seen as less of a priority compared to other aspects of service and care. This explains the relatively low acceptability rating of QIs describing aspects of patient experience and satisfaction.
In our environment, I think to have an expectation that you have this amazing customer service experience, amazing clinical care, and amazing outcomes. People are fallible and I don't think we can provide that every time. (Participant #1) Sub-theme 2b: Proxy measures. QIs describing aspects of patient satisfaction were seen as limited in their validity as measures of quality of prehospital care. Patient satisfaction metrics were seen to represent the patient's subjective contentment with the service, a distinct aspect of care. Patient satisfaction QIs were considered important patient-centred measures, but participants did not think that they should be used as a proxy for overall quality of prehospital care.
Sometimes we're unable to explain our thought processes to our patients, to a level that they understand, […]. I found that quality indicator maybe is very dependent on the patient. I don't know if it would work as well as the others. (Participant #3) Theme 3: Outcome indicators. Generally, the importance of patient outcomes and associated measurement was well recognised by participants. Structure and process type QIs were seen as more acceptable when they aligned with outcomes of interest. Nevertheless, some clinicians raised concerns about outcome indicators and their sensitivity to differences in prehospital quality of care. Prehospital care was described as a brief initial part of a much more extensive care pathway, especially in critically ill or injured patients. Outcome measurement at a distant point in that pathway may have limited ability to determine prehospital care quality. Theme 4: Time intervals. When asked about indicators which involve general time intervals, participants considered these to be less acceptable. QIs which were specific about time-sensitive patient cohorts (i.e., those with conditions requiring timely interventions not available in the prehospital setting such as endovascular thrombectomy, damage control surgery, or percutaneous coronary intervention) were seen as more acceptable. It was important to participants that any QI which included time-intervals was specific about critical interventions rather than less meaningful aspects, such as arrival on scene or nonspecific transport destinations. Any QI with time frames needed to be achievable and contextual, reiterating the importance of practicality.
There's no robust evidence that demonstrates that arriving at a scene within eight minutes of being dispatched improves anything. There's no evidence to support any of that. Theme 5: Professional values and qualities. As part of the a priori interview questions, participants were asked how the proposed QIs aligned with their professional values and qualities. Participants found it challenging to comment on this. Nevertheless, most participants said that the suite of QIs connected to what they believed to be their professional values and qualities.
'I don't quite know how to answer that, I'll be honest. But I think that I would say that they mostly aligned to most of the stuff that we want to do and that we should be aspiring to do'.

Discussion
Overall, participants found almost all QIs in the proposed suite to be acceptable. If a cut-off median score of 3.5 or greater in the ratings was applied, the initial list of 84 QIs would be reduced by only two. Nevertheless, besides commenting on desirable factors, participants also described aspects of the QIs that negatively affected the level of acceptability.
For participants in this study, the acceptability of the proposed QIs was dependent on the perceived presence of other key characteristics. When participants thought that a QI lacked clarity or when they believed a QI was poorly supported by evidence, they rated this QI as less acceptable. The need for a QI to possess clarity (a proxy for content validity) and be supported by evidence suggests a positive association between how acceptable QIs are to healthcare providers and scientific acceptability. Scientific acceptability addresses, at least in part, the basic measurement principle of validity. 44 Whilst the concept of validity is often applied to the results of measurement, it must also be considered within the elements of QIs and hence their development. In QI development, validity refers to the correctness of the QI as compared to scientifically credible sources. 5,44 Unsurprisingly, the findings of this study suggest that QIs which are developed systematically on the premise of being based on high-quality evidence, or a combination of best-available evidence and expert opinion when highquality evidence is scarce or absent, are more likely to be accepted by healthcare providers. The link between undisputable evidence and acceptability provides plausible explanation for some of the highest ratings, e.g., QI-A.4.3.
or QI-B.1.8. Since all QIs in this study were previously validated, this notion may also explain the high acceptability ratings in general.
Considerable commentary emerged about the practicality of what QIs describe. The acceptability of some QIs was rated relatively low by some participants because they felt that the described aspects were unrealistic, e.g., QI-A.6.2. and QI-B.6.4. Undoubtedly, this is one of this study's most important findings. Involving those who will be assessed and those who will conduct the measurement, analysis and resultant decision-making can provide a useful reality check for whether the QI is practical and sensible. Primarily this enables refinement of QIs and serves as a catalyst for their effective application. In other words, assessing the acceptability of QIs may increase their acceptability and hence successful use. 45 Linked to practicality of what QIs describe, participants also highlighted the importance of flexibility in QI application. The concept of variation may be considered in somewhat different ways. In performance measurement, variation often refers to changes in the data over time, its interpretation being one of the cornerstones of improvement science. 46 However, the variation that interview participants referred to is best described as a difference in healthcare processes, compared to peers or to a gold standard such as an evidence-based guideline recommendation, 47 or a QI. Variation is not automatically an indicator of poor quality. In fact, to some degree, variation should always exist because patients are unique and care should be responsive to differences between patients. 48,49 Participants raised concerns about inflexibility of QI application, and sensitivity to patient characteristics or situational demands not being recognised as warranted variation. Many participants highlighted that this may lead to 'pleasing the QI'. In other words, paramedics providing prehospital care that aligns with applicable QIs even when this compromises patient-centredness or other genuine aspects of quality.
Lastly within the 'key characteristics of QIs' theme, meaningfulness featured strongly in the interviews. For any indicator to be meaningful, it must have a relationship to the underlying phenomenon it is intending to signal. 50 Therefore, a prehospital care quality indicator must describe aspects of good patient care delivered by ambulance services. When theory or scientific evidence is robust, this link is clear. 51 However, in developing healthcare disciplines with a limited evidence base such as paramedicine, there may be dispute over the strength of a relationship between the proposed indicator and the underlying phenomenon to be measured. In the absence of an agreed definition of prehospital care quality, the acceptability of associated indicators will inevitably vary. This provides an explanation of the wide range of ratings for some of the QIs, e.g., QI-A.6.2. or QI-B.2.7. More importantly, it highlights the need for consensus on a definition of prehospital care quality as it would contribute to more consistent acceptance of aligned QIs and ultimately their strength in affecting meaningful improvement.
The first theme and its sub-themes reflect findings of similar studies investigating the acceptability of QIs in other healthcare disciplines. Burges and colleagues, 52 for example, found that QIs which were evidence-based and clearly worded were more acceptable to primary healthcare providers (family physicians, nurses and nurse practitioners) participating in their Delphi study. Increasingly QIs are described in terms of being fit-for-purpose and fit-for-use, together contributing to their actionability. A recent multiphase qualitative analysis exploring the concept of actionability of QIs. Barbazza, Klazinga and Kringos describe three clusters within which a QI's fitness for use can be appraised: methodological, contextual and managerial. 53 In particular, the methodological considerations resonate with the findings of this study supporting the idea that a QI which is systematically developed with careful considerations of key characteristics will be more acceptable to prehospital care clinicians and managers and ultimately possess more potential to facilitate improvement.
Different perspectives on health care quality often result in different expectations and thus different indicators of quality. As illustrated in sub-theme 1d, healthcare providers frequently view quality through clinical effectiveness and associated outcomes. Whilst outcomes are important to patients too, they frequently place extensive value on the emotional or interpersonal aspects of care. 54 This might be especially true in settings like prehospital care where noticeable outcomes are seldomly reached due to short patient contact times. As a result, participants considered QIs which described aspects of patient satisfaction to be of lesser priority and less valid as QIs of prehospital care. This does not mean that participants disregarded patient values or that QIs describing aspects of patient experience and satisfaction were unacceptable. It means that perspective matters and that the development of a symmetrical suite of QIs balancing different perspectives on quality will need to involve patients. 55 Whilst patient satisfaction is widely measured in healthcare, our finding indicates that it might still be a poorly understood concept among paramedics. This aligns with results of recent systematic reviews on the concept and determinates of patient satisfaction (irrespective of healthcare discipline) suggesting that there is a need for further research aiming to define patient satisfaction and how to validly utilise it as an indicator of healthcare quality. 54,56 In the evaluation of quality of health care, structural, process and outcome indicators all have advantages and disadvantages. 57 The perceived level of acceptability of outcome QIs was somewhat conflicted by the advantages and disadvantages of this type of indicator. Participants realised that outcome QIs are beneficial since they facilitate measurement of something that is important in its own right. However, since most outcome measurement will occur sometime after the brief prehospital care phase, it is reflective of all aspects of healthcare, not only that provided by paramedics. Outcome measurement is also influenced by variables other than healthcare processes, e.g., patient characteristics. Therefore, although generally recognised as important types of QIs, participants expressed concern about the attributability of outcome indicators in evaluating prehospital care quality. This explains the somewhat wider distribution in the acceptability ratings of QIs-B.2.8 to 10 and the statistically significant difference in medians between the different QI types. The problem of attribution in outcome measurement is not unique to prehospital care though. Attributing outcomes to the care and services provided by organisations or individuals in most healthcare discipline is often challenging due to the many factors that can influence a particular outcome. 58 Furthermore, the fact that outcomes may need a long time to manifest was recognised by Donabedian in his early conceptualisation of the various types of indicators. 59 Thus, similar concerns about the acceptability of outcome indicators were raise by primary healthcare providers in the study by Burge et al. where some participants felt that they had no power to influence outcomes and thus they considered them poor (and hence less acceptable) indicators of the quality of the care they provided. 52 Timeliness was considered to be an important attribute of prehospital care quality. Participants agreed that in timesensitive patients, such as cardiac arrest, stroke, or major trauma, timely access to healthcare contributes to desirable health outcomes, e.g., QI-B.2.3., QI-B.3.8. and QI-B.3.9. addl. However, participants reiterated what has been debated within the paramedicine discipline for some time there is little evidence to support the generic measurement of response times as an indicator of prehospital care quality. 60 It is worth noting at this point that in this project, indicators detailing general time intervals such as response time, time on scene, or turnaround time, were all deemed not valid in the preceding study. 18 Advances in ambulance deployment modelling and call triaging, and a systematically developed suite of QIs, should contribute to more sustainable performance and meaningful measurement of timely access to health care.
Participants found it difficult to comment on how the proposed QIs connected with their professional values and qualities. Similar to other registered healthcare professions, paramedicine in Australia is regulated by its own regulatory authority, the Paramedicine Board of Australia. The code of conduct for registered health practitioners was developed by most of the 15 National Boards. It states that "while individual practitioners have their own personal beliefs and values, there are certain professional values on which all practitioners are expected to base their practice". 61 (p. 6) The code describes a framework for the provision of appropriate, effective, and ethical health care. Thus, there should be a fundamental link between guidance on how to provide high-quality patient care and indicators thereof. Although hesitant, participants seemed to consider their professional values and qualities by reflecting on what is meaningful prehospital care and considered the suite of QIs to be in line. The alignment of QIs with professional values (and vice versa) is critical because the success of systems and processes to monitor, manage and regulate the quality of care provided to patients depends significantly on the values and behaviours of staff working throughout the system. 62

Limitations
The recruitment strategy includes a risk of selection bias. A range of strategies were considered and despite this limitation, our strategy still represented the optimal approach to attract participants. The formally calculated sample size (n = 149) for the survey was not reached. Therefore, Stage 1 is underpowered and at risk of type 2 error. Maintaining a confidence level of 95%, the sample size of 36 leads to a 16% margin of error signalling low confidence in the results of this study being representative of the Australian paramedic population. The results remain of value but should be considered as hypothesis generating rather than definitive. Limitations inherent to Likert-type questions may be present, namely central tendency bias, acquiescence bias, and social desirability bias; the latter two being more likely in light of the results. A "neutral" response could mean that participants were undecided on how to answer or that, despite previous work on optimising clarity as detailed in the Introduction, the particular QI was not clear to participants. Participants from NSW were overrepresented in Stage 1 of the study. Despite maximum variation sampling in Stage 2, the overrepresentation remained. Although semi-structured interview guides were used, the principal investigator's understanding and interpretation of the data may have potentially introduced confirmation bias. Furthermore, since the principal investigator conducted the interviews, he was not blinded to the study objectives, thereby potentially introducing bias. Determining the point of thematic saturation was based only on the information that was found, and on the judgement and experience of the researchers. Whilst multiple best practices for rigor in qualitative research were followed, transcripts were not returned to participants for checking of themes and subthemes. Further research is needed, which was beyond the scope of this study, to investigate how acceptable the proposed QIs are to patients and communities.

Conclusion
The findings of this study provide insight into how acceptable the proposed suite of QIs is to paramedics. More specifically, the results suggest that 82 of the 84 QIs may be acceptable to prehospital care providers. Through Stage 2 of this study, several attributes have been identified which could be transferrable to other prehospital care QI development initiatives. Through careful consideration of these, QI developers might ensure acceptability of their QIs from the start of the development process. Within this project, future research should evaluate the QIs' feasibility and reliability, investigate how acceptable they are to patients and communities, and look at ways to effectively implement the proposed suite of prehospital care QIs.