FormalPara Key Summary Points

Why carry out this study?

Patient-reported outcome (PRO) instruments such as the Retinopathy Dependent Quality of Life (RetDQoL) capture patient experiences of diabetic retinopathy (DR) but do not appear to adequately capture the patient experience of receiving treatment for DR, particularly proliferative diabetic retinopathy (PDR).

The aim of this study was to develop a de novo fit-for-purpose PRO instrument that comprehensively evaluates the patient experience of PDR.

What was learned from the study?

In this qualitative, mixed-methods study, the Diabetic Retinopathy-Patient Experience Questionnaire (DR-PEQ) assessed a wide range of symptoms, functional impacts, and treatment experiences in PDR.

Patients with PDR confirmed the comprehensiveness of the DR-PEQ and its relevance to their experience, and findings from preliminary psychometric analyses suggested it performed as intended.

The DR-PEQ is the first content-valid PRO instrument to comprehensively evaluate patient experiences in DR and its associated treatments.

Introduction

Diabetic retinopathy (DR) is a common vision-threatening complication of diabetes mellitus that substantially impacts patients’ daily lives [1,2,3]. DR was estimated to affect 103.1 million individuals globally in 2020, and its prevalence may increase by more than 50% by 2045 [3]. Moreover, the annual global incidence of DR ranged from 2.2% to 12.7% across population-based studies [4]. Severe visual impairment in advanced disease, or proliferative DR (PDR), significantly interferes with patients’ daily lives by limiting the performance of activities such as reading, driving, working, and recognizing faces [5,6,7,8].

DR is diagnosed by comprehensive eye examination and is characterized by microvascular abnormalities in the retina [1, 8, 9]. Microvascular damage induces upregulation of vascular endothelial growth factor (VEGF), a key mediator of angiogenesis [1, 9]. Increased VEGF expression promotes neovascularization, which can lead to substantial vision loss in PDR [1, 10, 11].

Treatment guidelines generally recommend close monitoring for early stages of DR without center-involved diabetic macular edema, whereas panretinal photocoagulation (PRP) or anti-VEGF therapy is recommended for the treatment of severe nonproliferative DR (NPDR) and PDR [8, 12]. PRP is a long-standing treatment for PDR given that it was shown to reduce the incidence of severe vision loss by at least 50% [13]. Anti-VEGF therapies including aflibercept [14] and ranibizumab [15] improved visual acuity, preserved retinal structure compared with PRP, and are recommended as an adjunctive or alternative treatment [8, 12, 16].

Although treatment efficacy in ophthalmology is primarily evaluated from the clinician perspective [13,14,15, 17], it is increasingly recognized that patients are best positioned to describe the impact of treatment on their disease [18]. Patients also provide unique and important insights regarding treatment effects on health-related quality of life (HRQoL) and other concepts that are distal to signs and symptoms of disease [18, 19]. Patient perceptions of symptoms or HRQoL can also complement observed clinician-reported outcomes and provide further evidence of treatment efficacy in clinical trials [18].

To date, few qualitative research studies have described the impact of DR and its treatment on HRQoL from the patient perspective [5, 20, 21]. Recently, a study was conducted in which a targeted literature search, online patient resources, and patient and clinician interviews informed the development of a conceptual model that described symptoms, their impact on daily function, and treatment experiences of patients with PDR [22]. This conceptual model illustrated a broad spectrum of symptoms and functional impacts that substantially affected patient HRQoL. For example, patients with PDR experienced many vision and eye problems that interfered with activities of daily living as well as emotional and social functioning.

Given that many symptoms and impacts in PDR are internal to patients and not easily observable by clinicians, the patient perspective is very informative in the evaluation of treatment response [18]. Patient-reported outcome (PRO) instruments provide key insights into how patients feel, function, and survive directly from patients without judgment or interpretation from a third party, allowing for the evaluation of clinically meaningful treatment outcomes [18, 23]. PRO instruments are also essential for assessing symptom burden and treatment monitoring [24], which is particularly important given that DR is a progressive disease [8]. The US Food and Drug Administration (FDA) in their 2009 PRO guidance emphasized the importance of the patient perspective, recommending that patient-reported experiences serve as the basis of PRO development [23]. This recommendation continues to be echoed by the FDA in the current patient-focused drug development series [25].

To determine whether existing PRO instruments adequately assessed PDR-related aspects of HRQoL in accordance with FDA guidance [23], five PRO instruments were identified from a search of published literature and clinical trial records: National Eye Institute Visual Function Questionnaire-25 [26], Impact of Vision Impairment [27], Retinopathy Dependent Quality of Life [28], Low Luminance Questionnaire [29], and the Retinopathy Computer Adaptive Test [30]. An extensive review revealed these PRO instruments failed to cover many concepts that were relevant for evaluating treatment effect, particularly in relation to treatment experience. The aim of this study was to address these gaps by developing a de novo fit-for-purpose PRO instrument that comprehensively evaluated the impact of treatment on symptoms, functional impacts, and HRQoL using existing knowledge of the patient experience of PDR.

Methods

Study Design

Development and preliminary validation of the PRO instrument is described in Fig. 1. To confirm relevance and assess performance of the instrument, patients with PDR who received aflibercept and/or PRP were recruited to participate in two rounds of in-depth, semi-structured interviews from November 2020 to January 2021 (Wave 1) and March 2021 (Wave 2) [22]. During interviews, trained researchers administered the PRO instrument to patients via phone. Following each round of interviews, items were revised on the basis of feedback regarding missing concepts and relevance, clarity, and suggested modifications. To inform the measurement continuum and ensure the instrument included all essential components, data from Wave 2 interviews were used to perform preliminary quantitative psychometric analyses using Rasch measurement theory (RMT). Feedback from Wave 2 interviews was used to further refine the PRO instrument and develop the corresponding conceptual framework.

Fig. 1
figure 1

Development and preliminary validation of PRO instrument. PRO patient-reported outcome, RMT Rasch measurement theory

Full size image

Stage 1: Development of Diabetic Retinopathy-Patient Experience Questionnaire (DR-PEQ)

Item Generation

To design a PRO instrument that evaluated clinical benefit following treatment, item generation focused on concepts that were considered most proximal to patients such as symptoms and functional impacts (e.g., on activities of daily living) given that these concepts were the most relevant for evaluating treatment effect based on clinical hypothesis. Specific attention was given to perceived vision problems and their impact on daily activities. Items were also developed to assess emotional and social impact. Items were included in the preliminary DR-PEQ, in which patients were asked to indicate the level of difficulty with daily activities (no difficulty at all, a little difficulty, some difficulty, a lot of difficulty, or could not do at all) and the frequency of emotional impacts, social impacts, and vision problems attributed to DR and its treatment (none of the time, a little of the time, some of the time, most of the time, or all of the time) in the past 7 days.

Stage 2: DR-PEQ Content Validity Evaluation

Participants

From October 2020 to January 2021, patients in the New Orleans metropolitan area were recruited by a healthcare market research firm using resources such as patient associations and social media. All eligible patients were at least 18 years old with clinician-confirmed diagnoses of diabetes mellitus and PDR, had received aflibercept or PRP within 6 months of study initiation, and were able to read questions on paper or a screen. Patients who participated in Wave 1 interviews were invited to participate in Wave 2 interviews upon completion of the Wave 1 interview analysis. Full exclusion criteria, recruitment methods, and sampling were previously described [22]. A recruitment target of 30 patients was established to allow for RMT analyses to be conducted [31]. Additional details are provided in Supplementary Methods.

Ethics

This study was conducted in accordance with the principles of the Declaration of Helsinki and all applicable regulatory requirements. All study documents including the protocol, screener form, interview guides, demographic and health information form, and informed consent form were approved by the Western Institutional Review Board-Copernicus Group Independent Review Board (study number 20202896) before study initiation. All patients provided written informed consent before participation in the study, and compensation was provided for participation in interviews.

Qualitative Patient Interviews

Patient interviews (ca. 1 h) were conducted via phone by three experienced qualitative researchers (JB, CS, AL) with backgrounds in health psychology who received study-specific training in conducting mixed concept elicitation and cognitive debriefing interviews using open-ended questions in a semi-structured format and the “think aloud” process to elicit spontaneous patient feedback. The interviewers were unknown to the patients prior to interviews and were not involved in any aspect of the patients’ healthcare. Standard qualitative methods were used to conduct all patient interviews, and concept elicitation was conducted in both rounds as previously described [22]. Concept elicitation consisted of open-ended questions that allowed patients to describe their experience with PDR spontaneously and thoroughly in their own words.

Cognitive debriefing comprised a “think-aloud” examination of the DR-PEQ in which patients verbalized their thought process and provided a response to each item (e.g., “Tell me what you are thinking about…” or “How are you coming up with your answer to the question?”) [32,33,34]. To confirm that patients’ interpretation of each question was consistent with the intended meaning, patients were asked to describe their understanding of the instructions and listed concepts and to confirm the clarity and relevance of items provided in the questionnaire. Patients were also asked to confirm their understanding of the response options for the difficulties they experienced in daily activities and to confirm whether any items or response options should be added or removed.

Coding Analysis

Transcripts were analyzed using thematic analysis and coded using an open and inductive coding approach in ATLAS.ti software (ATLAS.ti GmbH, Berlin, Germany) [35,36,37] as previously described [22]. During cognitive debriefing, patient feedback was coded and analyzed to identify any issues relating to relevance, clarity, interpretation, acceptability/appropriateness, conceptual overlap, time frame, response options, missing concepts, and meaningful change. The methodology of coding and analysis was identical in both rounds of interviews.

RMT Analysis

In exploratory RMT analyses, observed data from the Wave 2 patient sample were compared against RMT criteria to assess the following psychometric properties in an RMT model: scale-to-sample targeting, the measurement continuum, and sample measurement. RMT analysis was performed using RUMM2030 software (RUMM Laboratory, Perth, Australia). Additional details are provided in Supplementary Methods.

Results

Sample Characteristics

Forty patients participated in Wave 1 interviews (Table 1). The mean (standard deviation, SD) age was 53.7 (14.7) years. Most patients were male (n = 24 [60.0%]) and Black or African American (n = 21 [52.5%]), and no patients were Hispanic or Latino. Mean (SD) time since diagnosis of diabetes and PDR was 23 (9.0) and 6 (7.3) years, respectively. Furthermore, 34 (85.0%) and 23 (57.5%) patients previously received PRP and aflibercept treatment, respectively. Of the 40 patients who completed Wave 1 interviews, 30 (75.0%) patients completed Wave 2 interviews. Among these patients, the mean (SD) age was 53.0 (14.5) years. Although the proportions of male and female patients were comparable (n = 15 [50.0%]), most patients were Black or African American (n = 16 [53.3%]) as was observed in the Wave 1 population. Mean (SD) time since diagnosis of diabetes and PDR was 24 (7.0) and 8 (8.0) years, respectively. Most patients in Wave 2 previously received PRP (n = 26 [86.7%]) and aflibercept treatment (n = 19 [63.3%]). Hypertension was the most common comorbidity in both Wave 1 and Wave 2 populations (n = 34 [85.0%] and n = 25 [83.3%], respectively).

Table 1 Patient demographic and clinical characteristics
Full size table

Development of Preliminary DR-PEQ

DR symptoms, impacts, and treatment experiences that were identified from a previous literature search [22] contributed to the generation of the preliminary DR-PEQ, which comprised 72 items that spanned four scales: Daily Activities (52 items), Emotional Impact (3 items), Social Impact (3 items), and Vision Problems (14 items).

Cognitive Debriefing of Preliminary DR-PEQ

Patients reported that instructions, items, and response options of the preliminary DR-PEQ were easy to understand and relevant during Wave 1 interviews. Patients suggested several revisions to improve clarity and interpretation, including specifying which eye the items were inquiring about, providing greater specificity for several items of the Daily Activities scale (e.g., providing examples of “doing fine work with your hands”), and clarifying that items were asking about visual symptoms and impacts rather than other physical or mental symptoms and impacts. Patients noted several conceptual overlapping items (e.g., between “reading” and items related to fine print, smart phone, computer screens) that presented opportunities to minimize redundancy. Patients also reported that some items were generally irrelevant (e.g., “handling cash”) or were irrelevant in the context of COVID-19 (e.g., “moving around in a dark theater” and the entire Social Impact scale).

Patients supported the addition of a new response option for the Daily Activities scale that allowed them to indicate when an activity had not been performed in the past week. Without this response option, patients felt obligated to choose the response option “no difficulty” for such activities, which may have led to false floor effects for this scale.

Revisions to Preliminary DR-PEQ

Based on concept elicitation during Wave 1 interviews, four new items were added to both the Daily Activities and Vision Problems scales, and one item was added to the Emotional Impact scale. A new five-item Treatment Experience scale was also included in the DR-PEQ. Following cognitive debriefing, seven items and the Social Impact scale were removed because of conceptual overlap, irrelevance, or misinterpretation. Furthermore, several items and instructions were edited to improve clarity and facilitate interpretation, and a response option was added to the Daily Activities scale to allow patients to indicate they had not performed an activity within the recall period.

Cognitive Debriefing of Revised DR-PEQ

Patients from Wave 2 interviews considered the revised DR-PEQ to be relevant and comprehensive. Patients provided feedback on opportunities for greater clarity and improved interpretation, mainly by requesting examples or clarification of activities in the Daily Activities scale. Researchers in this study reported that some items may have been misinterpreted because of attribution to physical rather than visual symptoms of diabetes (e.g., neuropathy).

Patients recommended additional modifications to the Daily Activities scale. Some items were irrelevant or overlapped with other items (such as “paying with a credit card in person” and “going to medical appointments,” respectively). Patients asked for further clarification on “performing other usual leisure activities” and “driving in familiar and unfamiliar places” and noted the level of difficulty associated with certain activities varied on the basis of environmental conditions (e.g., amount of light available or familiarity with surroundings). Patients also suggested to add items related to navigating plane travel, getting off public transport, and putting on jewelry.

The Emotional Impact scale was well understood by patients. Patients suggested to add an item related to the emotional impact of treatments (e.g., “feel afraid or anxious about your diabetic retinopathy treatment”).

When patients reviewed the Vision Problems scale, the item “flashes of light” was misinterpreted. Patients also reported that “distinguishing different levels of contrast (e.g., seeing different shades of gray)” was excessively wordy. As was requested for the Daily Activities scale, patients asked for a response option indicating they had not experienced any vision problems within the recall period.

During assessment of the Treatment Experience scale, patients provided different responses to items based on their experience during versus after treatment. Patients misinterpreted “temporary increase in vision problems after treatment” as a temporary increase in vision following treatment (i.e., symptomatic improvement).

RMT Analysis

When scale-to-sample targeting was assessed, the Daily Activities and Treatment Experience scales demonstrated excellent coverage (96% and 100%, respectively) of sample measurements (Fig. S1 in the Supplementary Material). The Emotional Impact and Vision Problems scales demonstrated good (80%) and very good (89%) coverage, respectively. Item thresholds, item fit, and item dependency were evaluated in each scale to examine the measurement continuum. When item thresholds were assessed, no items of the Daily Activities and Vision Problems scales displayed disordered item response thresholds (Fig. 2a, c). However, one item of the Emotional Impact scale and two items of the Treatment Experience scale displayed disordered item response thresholds (Fig. 2b, d).

Fig. 2
figure 2

a Daily Activities, b Emotional Impact, c Vision Problems, and d Treatment Experience item thresholds. The x-axis represents the measurement continuum of the construct under measurement, with decreasing levels of difficulty from left to right. The y-axis shows each of the item response categories for each response option. Response categories are shown using colored blocks that contain successive integer scores (e.g., 0, 1, 2, 3, 4). Thresholds for items are missing and replaced with ** if they are disordered, i.e., response categories do not appear in a consecutive increasing order in relation to the construct (x-axis)

Full size image

Item fit was very good across all four scales (Tables S1–4 in the Supplementary Material). When item dependency was examined, 93 item pairs of the Daily Activities scale were shown to influence each other. In contrast, no items of the Emotional Impact scale were found to be dependent on each other. The Vision Problems and Treatment Experience scales had nine item pairs and one item pair, respectively, that influenced each other.

When the sample measurement was assessed, the Daily Activities scale had excellent reliability (estimated person separation index [PSI] 0.95). The Emotional Impact and Vision Problems scales had good reliability (estimated PSI 0.81 and 0.76, respectively), and the Treatment Experience scale had reasonable reliability (estimated PSI 0.64).

Refinement of DR-PEQ

Revisions based on Wave 2 interviews informed the generation of the current DR-PEQ, which consists of 85 items across four scales: Daily Activities (54 items), Emotional Impact (5 items), Vision Problems (15 items), and Treatment Experience (11 items). The current conceptual framework is presented in Fig. 3.

Fig. 3
figure 3

Conceptual framework of the DR-PEQ. DR-PEQ Diabetic Retinopathy-Patient Experience Questionnaire, PDR proliferative diabetic retinopathy, PRO patient-reported outcome

Full size image

Discussion

Given that significant conceptual gaps were identified in existing PRO instruments used in PDR, the aim of this study was to develop a de novo fit-for-purpose PRO instrument that evaluated the effect of treatment on symptoms, functional impacts, and HRQoL in patients with PDR. The DR-PEQ, an 85-item PRO instrument, was developed focusing foremost on content validity by using the conceptual model of patient experiences in PDR, consistent with FDA guidance [23, 25]. This study provided preliminary evidence that the DR-PEQ assessed symptoms, impacts, and treatment experiences that are relevant to patients with PDR and demonstrated favorable psychometric properties. In contrast to previously developed PRO instruments, the DR-PEQ also evaluated potential side effects and treatment complications that may further impact patient function and well-being. The DR-PEQ is therefore the first content-valid PRO instrument to assess in granular detail the effects of treatment on outcomes that are relevant to patients and may complement clinician-reported outcome measures in the assessment of clinical benefit following treatment in clinical trials, real-world settings, and patient registries.

A broad range of symptoms, impacts on daily living, and treatment experiences were adequately captured by the DR-PEQ. Items of the Daily Activities scale represented a continuum of visual functioning, with the highest-functioning patients experiencing difficulties with reading and driving in suboptimal conditions and the lowest-functioning patients experiencing difficulties with activities under optimal conditions. The DR-PEQ assessed various aspects of emotional and social functioning as well as common treatment-related complications patients may experience. The DR-PEQ therefore may effectively describe a wide range of patient experiences that may change with treatment over time.

Cognitive debriefing of the DR-PEQ showed most items were relevant, well understood, and easy to answer. Minor changes were implemented on the basis of patient feedback, including the addition of appropriate response options for patients who had not performed an activity or experienced any problems. Patients confirmed that items were relevant to their experience and that the 7-day recall period was appropriate. These findings were supported by the RMT analysis, which provided preliminary evidence the questionnaire was performing as intended.

The DR-PEQ encompasses a broad spectrum of patient experiences in PDR, allowing for the identification of many physical, emotional, and social aspects of patients’ lives that may be improved with treatment. PRO instruments such as the Visual Function Questionnaire-51/-25 [26, 38] and short-form questionnaires (e.g., SF-12) [39] have been used to assess HRQoL in DR, but these instruments were not tailored to evaluate treatment effect, symptoms, or functional impairments that may be specific to DR. In contrast, the DR-PEQ evaluates DR symptoms and impacts that matter most to patients, ultimately empowering them to contribute to their care and provide input that will promote informed treatment decision-making [24]. Although the RetDQoL and RetCAT were developed to evaluate HRQoL in patients with DR [28], the RetDQoL does not evaluate potential effects of treatment on HRQoL [28] and the RetCAT, a computerized adaptive test [30], may not be suitable for use within the context of a clinical trial. The DR-PEQ comprehensively assesses the impact of symptoms and DR-associated treatment on HRQoL, supporting a holistic evaluation of the patient experience in PDR. The final DR-PEQ may include electronic and interviewer-administered components, which will accommodate patients in clinical trials and real-world settings. The DR-PEQ may also support the evaluation of additional clinical benefit provided by new DR treatments.

This study is not without limitations. Assessment of psychometric properties was limited to a small patient sample that did not include Hispanic/Latino or Asian patients, limiting the generalizability of study findings. Moreover, test–retest reliability, construct validity, and sensitivity to change were not formally investigated in this study. Future studies are warranted to further evaluate psychometric properties of the DR-PEQ, validate the DR-PEQ in a larger and more diverse patient population, and confirm its appropriateness for the assessment of treatment effects cross-sectionally and longitudinally in patients with PDR. Given that factors such as treatment durability, accessibility, and frequency may also impact the experience of patients with PDR, these aspects of treatment could be explored in subsequent studies. The performance of the DR-PEQ must also be evaluated in clinical trials and real-world settings.

Several potential modifications could be considered to optimize the DR-PEQ further. The potential modifications would need to be considered within the context of the objectives of the study. For example, specific items may be added to improve the precision of measurement in patients with relatively mild PDR symptoms. Moreover, the Treatment Experience scale may be expanded following further research to include items that address other areas of surgery and ophthalmology, in general. Shorter versions or a modular approach may also be applied to decrease respondent burden.

Conclusion

This study supports the content validity of the DR-PEQ, developed on the basis of FDA guidance [23, 25], to characterize in detail the impact of treatment on symptoms and HRQoL in patients with PDR. Additional studies are required to further validate the DR-PEQ and assess its performance in a large patient population. Following further psychometric validation, this PRO instrument may be integral for providing important insights into the patient experience with PDR, enabling healthcare providers to better understand the burden of PDR on patients and to assess clinically meaningful changes following treatment.