Reconciling validity and challenges of patient comfort and understanding: Guidelines to patient‐oriented questionnaires

Abstract Background Patient‐reported outcome measures (PROMs) are widely recognized as important tools for achieving a patient‐centred approach in health research. While PROMs are subject to several stages of validation during development, even questionnaires with robust psychometric properties may challenge patient comfort and understanding. Aim Building on the experience of patient engagement in the PriCARE research programme, this paper outlines the team's response to concerns raised by patient partners regarding the administration of the questionnaire. Methods Based on a participatory action research design and the patient engagement framework in the Strategy for Patient‐Oriented Research of the Canadian Institutes of Health Research, PriCARE team members worked together to discuss concerns, review the questionnaires and come up with solutions. Data were collected through participant observation of team meetings. Results This paper demonstrates how patient partners were engaged in PriCARE and integrated into the programme's governance structure, focusing on the challenges that they raised regarding the questionnaires and how these were addressed by PriCARE team members in a six‐step approach: (1) Recognizing patient partner concerns, discussing concerns and reframing the challenges; (2) Detailing and sharing evidence of the validity of the questionnaires; (3) Evaluating potential solutions; (4) Searching the literature for guidelines; (5) Creating guidelines; and (6) Sharing and refining guidelines. Conclusion This six‐step approach demonstrates how research teams can integrate patient partners as equal members, develop meaningful collaboration through recognition of individual experiences and expertise and ensure that the patient perspective is taken into consideration in research and healthcare innovation. Patient or Public Contribution All patient partners from the PriCARE programme were actively involved in the six‐step approach. They were also involved in the preparation of the manuscript.


| INTRODUCTION
It is now widely recognized that involving patients in healthcare research helps to capture what is most important to them. Patientreported outcome measures (PROMs) are one of the primary tools in the philosophy of a patient-centred approach to assess care. They highlight what really matters in the patient's life and can measure the impact of an intervention from the patient's perspective. 1 As stated by international guidance, questionnaires of PROMs are subject to several stages of validation, such as identifying the conceptual framework of the measure and assessing validity and fidelity. 2,3 During the process of validation, the FDA 2 and the International Society for Quality of Life Research 3 recommend involving patients in the development of the PROM, for example, by interviewing patients to establish or confirm its face validity. Still, there remains a large variation in the levels and methods of patient engagement. [4][5][6][7][8] Despite recommendation from the scientific community to avoid the reformulation of questions after a PROM has been validated, 9,10 even questionnaires with robust psychometric properties may pose challenges for vulnerable populations.
These challenges can include lack of clarity, degree of comfort answering certain questions or the applicability of a generalized questionnaire to the particular, complex situation of the individual. The language used, or the way in which the questions are presented, may not always be clear to patients, 11,12 and the collection of demographic data may make patients uncomfortable. 13 Questionnaires may be challenging for certain vulnerable groups, such as people with cognitive impairments, multimorbidity or low literacy levels. For people with cognitive impairments, quantitative evaluation, comparison questions, abstract concepts and generalized statements can be challenging. 14 The unique situation of patients with multiple chronic conditions is not always taken into account in the way in which questions are formulated. 6 Individuals belonging to some vulnerable populations may have difficulty in understanding rating scales and mutually exclusive response choices or may find them too complex. [15][16][17][18] Rigid adherence when administering questionnaires to older people may inhibit the interviewer from interacting and engaging meaningfully with the respondent or responding to cues indicating distress or grief. 17 To overcome some of these challenges, Boynton et al. 16 propose a guide to questionnaire research for 'reaching beyond the white middle classes'. For example, they suggest placing demographic questions at the end of the questionnaire to minimize the threat for participants.
Further, they emphasize the importance of explaining clearly why the information is required and how it will be protected. They also suggest working closely with community representatives and conclude that the quality of social interactions during questionnaire administration undoubtedly influences responses.
The inclusion of patients as genuine partners on research teams requires researchers to be open and responsive to their feedback.
Building on the experience of the PriCARE team, this paper proposes (1) steps to address the challenges of patient comfort and understanding of the questionnaires and to reach consensus as a team, and (2) patient-oriented guidelines for administering the questionnaires.

| The PriCARE research programme
The PriCARE research programme 19,20  The PriCARE research programme is a multiple-case embedded mixed-methods study conducted in five participating Canadian provinces: Newfoundland and Labrador, Nova Scotia, New Brunswick, Quebec and Saskatchewan. One or two primary care clinics were recruited in each province to implement and evaluate the CM intervention among frequent users of healthcare services with chronic diseases and complex care needs. The intervention is detailed elsewhere. 19 To fulfil the inclusion criteria for the programme, participants must be over 18 years of age, living with at least one chronic condition (including mental health), have complex healthcare needs, frequent users of healthcare services according to professional judgement and likely to benefit from the intervention. Patients with loss of autonomy, living in a long-term care residence, having a prognosis of less than one year or unable to consent were excluded.
In the PriCARE research programme, we conducted a participatory action research project in which both academic and nonacademic researchers collaborated closely. 21 This design focuses on enabling action through a reflective process, in which stakeholders participate in the data collection and analysis, and identify areas of action in collaboration with academic researchers. 22 Participatory research aims to improve the participation of the population for whom the research results are intended, to mitigate power imbalance and to recognize and promote the value of lived experience. In Pri-CARE programme research, researchers (principal and coinvestigators, research assistants and a postdoctorate fellow) work closely with patient partners.
In line with the SPOR of the Canadian Institutes of Health Research, PriCARE programme research actively engages patient partners as equal team members in all aspects of the research process. 23,24 A community of patient partners was formed, including six patient partners, one or two per province, who worked closely with the re-

| The patient questionnaires
The PriCARE research programme uses a mixed-method data collection approach, including patient questionnaires. Based on a literature review, instruments were carefully selected to measure several variables, such as health literacy, 25,26 multimorbidity, 27,28 care integration, 29,30 selfmanagement, 31,32 quality of life 33,34 and psychological distress. 35,36 Inclusion criteria were as follows: ability to be self-administered, good psychometric properties, availability in English and French and low number of items. All instruments selected were short (≤21 items), reliable and valid in both languages. See the Supporting Information Appendix S1 for more details.

| The challenges with the questionnaires
During a meeting with a research assistant, patient partners reviewed the patient questionnaires, which were comprised of several prevalidated instruments (Supporting Information Appendix S1), and expressed reservations regarding the general administration of the questionnaires, as well as with specific questions. First, patient partners indicated that going through the questionnaires was an uncomfortable experience. They emphasized the personal nature of some of the questions, noting that certain questions could be triggering as a result of past negative healthcare experiences, and being asked to reflect on difficult topics. These topics included their ability to care for themselves, their experiences with physical or psychological distress as a result of multiple chronic illnesses and mental health challenges. Patient partners emphasized the importance of establishing a relationship of trust between study representatives and participants by starting the administration process slowly, reviewing the purpose of the questionnaire, and easing into more difficult questions to help alleviate discomfort.
Second, patient partners desired clarification on why participants were being asked such questions, and whether, or how, their responses would be beneficial. They felt that further justification was needed for particularly sensitive or invasive questions for participants to feel comfortable sharing their responses.
Third, patient partners found some of the questions confusing or found the wording difficult to understand. They felt that there were several ways to interpret these questions, and that participants would need assistance, as the response choices did not always reflect the type of answer that participants would have thought to provide.
Fourth, patient partners felt that some of the questions were phrased in a way that presumed that the respondent had a single health condition and did not adequately reflect the realities of living with multiple chronic conditions. For example, when considering a question concerning health literacy, patient partners noted that some of the respondents' illnesses may be well managed by the participant, while others may be more challenging and less understood. Thus, patient partner participants would find it difficult to answer questions that did not account for multiple, complex needs. Patient partners expressed interest in adding, removing or revising some of the questionnaire items and felt that some of them seemed outdated. It is important to note that, in general, many patients suffer from multiple chronic conditions, often alongside mental health conditions and/or social vulnerabilities, 37,38 which may interfere with their ability to complete a questionnaire. 16 These concerns were discussed during a PriCARE team meeting and the team members decided to find solutions to address these concerns.

|
Step 2: Detailing and sharing evidence for the validity of the questionnaires A table, similar to the Supporting Information Appendix S1, detailing the evidence of the instruments' validity and supporting their use was presented to PriCARE team members to support the discussion of potential solutions. For each variable, the PriCARE team members discussed whether other instruments would be more appropriate.
Literature searches were undertaken to identify potentially relevant instruments. At the end of this process, based on the criteria for selecting tools, the PriCARE team members concluded that initial instruments were appropriate for the research programme.

| Step 3: Evaluating potential solutions
Despite the above-mentioned concerns raised by the patient partners, the possibility of replacing or introducing significant modifications to the questionnaires was discarded by researchers as this could compromise the psychometric properties of instruments that had already been formally assessed for reliability and validity. 15 However, it was obvious that the PriCARE team members could not use the questionnaires without introducing minor modifications, such as the inclusion of techniques to promote patient comfort and understanding. Indeed, the ability to establish rapport with patients is one of the most important skills that researchers need to enable engagement and conduct a sensitively presented interview. This rapport can be developed by a range of researcher behaviours, such as paying special attention to the patient, finding common ground, being courteous and empathic, making jokes and sharing personal information when appropriate and providing emotional support. Researcher-administered questionnaires, rather than self-administered questionnaires, were encouraged to build this rapport with the patient. 41 The PriCARE team members also favoured the use of neutral statements and probes that would not affect the validity of the measures (e.g., by saying: there are no right or wrong answers), interviewing techniques to encourage response to difficult questions (e.g., by providing clarification to stimulate a response) and reducing the burden of the patient by considering some of the challenging aspects of the questionnaires (to reduce fatigue, consider sensitivity to triggers, distress). 42

| Step 4: Searching the literature for guidelines
The need to adapt questionnaire items for vulnerable and diverse populations has been noted in the literature 15  Boynton et al. 16 propose that representing 'disempowered and socially excluded groups, cross cultural issues, and participants whose physical and mental health may interfere with their ability to complete a questionnaire' may be promoted through careful administration of questionnaires in addition to training and support of research staff. The administration of questionnaires is seen as a social interaction that may be challenging for both the participant and the researcher, 13 and as such, must be rethought to recognize and overcome their inherent bias towards White, well-educated populations and to adapt them to the needs of larger segments of the population. 13,15 The World Health Organization regularly develops guidelines for administration to accompany survey instruments. 43

| Patient-oriented guidelines for administration of questionnaires
The primary modification to the PriCARE patient questionnaires was the creation of a manual to guide the administration of the PriCARE questionnaires. Researcher-administered questionnaires are intended to facilitate participant completion, but are considered more labour intensive. 18  The second modification to the questionnaires was a format modification 15 where clarifications were added to challenging questions to reduce respondent burden and enhance comprehension and motivation to complete the questionnaires. 15 Clarifications are required in situations where the participant is unable to understand the question or the response choices, or expresses difficulty or confusion regarding a questionnaire item. 42 Short statements were added next to challenging questions, and interviewers were instructed in the guidelines only to use them after neutral probes or feedback failed to stimulate a response.

| DISCUSSION
This is the first study to report the process of researchers and patient partners working together to address challenges of patient comfort and understanding of the questionnaires and to propose patientoriented guidelines for administrating the questionnaires. Although the World Health Organization 42 proposed a guide for administration to accompany survey instruments, their guidelines were developed based on the scientific expertise of academic researchers and did not involve patients.

| Process and guidelines adaptable for patientoriented projects
With the growing interest worldwide in PROMs, 1,44-46 optimizing patient understanding and comfort during the administration of questionnaires while ensuring validity and respect of psychometric properties becomes an important target to be achieved. Since the balance between comfort and validity remains challenging, 15,16,18 the sixstep approach to reach consensus with patient partners and the guidelines presented in this paper will be useful for research teams, especially those working with vulnerable populations such as people with chronic conditions, complex needs, cognitive impairments or belonging to minority or migrant groups, highlighting that collaboration.

| CONCLUSIONS
The process of reaching a compromise about questionnaire administration-retaining validated instruments but reviewing their administration-is a powerful demonstration of genuine, collaborative partnership between patients and researchers. This six-step approach demonstrates how research teams can integrate patient partners into the research team as equal members into the research team and develop meaningful collaboration through recognition of individual experiences and expertise. Ultimately, incorporating these steps ensures that the patient's perspective is expressed through questionnaire research, the development of data collection tools and healthcare innovation in general.

ACKNOWLEDGEMENTS
We would like to thank members of the PriCARE team, the principal investigators, coinvestigators, coordinator and research assistants, case managers and patient partners. This study is supported by the

DATA AVAILABILITY STATEMENT
The data sets supporting the conclusions of this article are included within the article and its additional files.