The diabetes management experiences questionnaire: Psychometric validation among adults with type 1 diabetes

To examine the psychometric properties of the Diabetes Management Experiences Questionnaire (DME‐Q). Adapted from the validated Glucose Monitoring Experiences Questionnaire, the DME‐Q captures satisfaction with diabetes management irrespective of treatment modalities.


| INTRODUCTION
With type 1 diabetes management, treatments and technologies advancing rapidly, it is crucial to assess the satisfaction and experiences of those who use them in their everyday lives.There are recognized associations between people's experiences of treatments and technologies, their continued and optimal use and their consequent benefits. 1he user's satisfaction with their diabetes treatment and/ or technology is a key determinant of continued/optimal use of that treatment or technology. 2 For the Australian Hybrid Closed Loop (HCL) trial, 3 there was a need to assess experiences and satisfaction with current diabetes management, regardless of the method used, which varied across participants by timepoint and trial arm allocation.At baseline, everyone used multiple daily injections (MDI) or insulin pump, but not continuous glucose monitoring (CGM), prior to half of eligible participants being allocated, at random, to HCL or to continue with their current treatment.
Several validated questionnaires have been used to assess the satisfaction or experiences of adults with type 1 diabetes using modern glycaemic technologies, for example, insulin delivery devices, glucose monitoring devices, sensor-augmented pumps and hybrid closed loop. 4,5Existing satisfaction measures were considered inappropriate for this study, either due to their focus on a single mode of insulin administration, 6,7 or on glucose monitoring. 6,8,9thers were considered inappropriate due to their brevity, broad focus and age (pre-dating modern devices).For example, the total score of the Diabetes Treatment Satisfaction Questionnaire 10 is calculated by summing six items, three of which focus broadly on the outcome or appraisal (i.e.current satisfaction, willingness to continue and willingness to recommend), with only three single items focused on dimensions of satisfaction, for example, convenience, flexibility and understanding, which determine the outcome/appraisal.Thus, the DTSQ has limited value for understanding both the dimensions or determinants of satisfaction with advanced technologies. 11In recent years, questionnaires have been developed specifically to assess experiences with automated insulin delivery systems. 12,13owever, such measures were not available in time for the Australian Hybrid Closed Loop (HCL) trial, 3 and may not have been suitable for use at baseline by those using MDI.Thus, another measure was needed.
The Glucose Monitoring Experiences Questionnaire (GME-Q) was designed for adults with type 1 diabetes to assess their satisfaction with glucose monitoring, regardless of their device type. 8Its development was informed by the literature, and by exploratory and cognitive debriefing interviews with adults with type 1 diabetes.It has a strong conceptual framework, with satisfaction considered the product of users' perceptions of effectiveness, convenience and intrusiveness, 14 confirmed through principal component analysis.Acceptability, convergent and divergent validity have also been established. 8All 22 items are relevant not only to glucose monitoring but also to the broader task of managing diabetes, and the three latent psychological constructs underpinning the GME-Q's structure are supported by previous research. 15,16Therefore, the GME-Q was adapted (described below) to assess satisfaction with diabetes management, for use in the Australian Hybrid Closed Loop study.
The three aims of this study were to examine the Diabetes Management Experiences Questionnaire (DME-Q) in terms of: 1. acceptability, validity and reliability; 2. responsiveness, hypothesizing that the DME-Q subscales and composite score would show significant differences at 26-week follow-up between the HCL and standard therapy (MDI or pump without real-time CGM) group, when adjusted for baseline; and 3. relationship with other psychological and clinical variables.

| Participants and procedure
Data were derived from a 26-week randomized controlled trial investigating the effectiveness of hybrid closed loop (HCL) insulin delivery (MiniMed 670G) compared to current diabetes treatment (ACTRN12617000520336).The protocol is published. 3Ethics approval was granted by the Human Research Ethics Committee at St Vincent's Hospital, Melbourne (HREC-D 088/16).Participants gave consent and clinical governance was overseen by the ethics committees of each of the participating institutions.
Eligible participants were adults (aged 25-70 years) with type 1 diabetes (recruited from seven Australian university hospitals), with glycated haemoglobin (HbA1c) ≤91 mmol/mol (≤10.5%)who had been using MDI or insulin pump without CGM for at least 3 months.
Participants completed an online survey (Qualtrics © 2015, Provo, UT) on a tablet computer when they attended the trial centre at three timepoints: For eligibility and clinical assessments prior to randomization (i.e.pre-randomization), and for the 13-week (midpoint) and 26-week (study end) assessments.The DME-Q is based on the content and format of the 22item GME-Q. 8The GME's introduction and item stem, which focus the respondent on 'my current method of monitoring …' were adapted to focus the respondent on 'my current method of managing my diabetes…'.The latter is a more holistic assessment designed to enable the respondent to consider what it takes to manage type 1 diabetes, which is necessarily broader than the former, which is limited to glucose monitoring alone.The introduction

Novelty statement
• With rapidly advancing type 1 diabetes treatments and technologies, it is crucial to assess satisfaction and experiences of those who use them in their everyday lives. of the questionnaire clarifies that 'diabetes management' could include the following: taking tablets or insulin injections or pump, checking glucose levels (e.g.finger pricks, continuous glucose monitoring) or using a closed loop system.The wording of all 22 original items (e.g.'…is convenient', '…gives me too much information') was considered relevant to the broader task of managing diabetes and remained unchanged.Each item is rated on a 5-point scale (1 = strongly disagree to 5 = strongly agree).A final, single overview item assesses overall satisfaction with the person's current method of managing diabetes, with the same response options (Table 2).The expected factor structure was that the 22 items would form three subscales, 'Convenience', 'Effectiveness' and 'Intrusiveness', as well as loading on a 'Total satisfaction' scale.For the three subscales, composite scores are calculated by averaging relevant item scores, to create a score ranging from 1 to 5. Scores for item 2 are reversed before contributing to the 'Convenience' subscale.Similarly, for the 'Total satisfaction' scale, item 2 and all 'Intrusiveness' item scores are reversed, summed and divided by the total number of items to create a total score from ranging 1 to 5. For all (sub)scales, higher scores indicate greater endorsement of the concept measured.
2.2.2 | Measures for the psychometric validation of the DME-Q The Diabetes Treatment Satisfaction Questionnaire (DTSQs) 10 assesses satisfaction with diabetes treatment over the past few weeks, on a 7-point scale (e.g.0 = very dissatisfied to 6 = very satisfied).The 'Total satisfaction' score is calculated by summing responses of six items (range: 0-36), with higher scores indicating greater treatment satisfaction.The remaining two items, assessing perceived frequency of hyperglycaemia and hypoglycaemia are reported separately (0 = none of the time to 6 = all of the time).The Problem Areas in Diabetes (PAID) 17 scale includes 20 items describing diabetes-related problems.The respondent indicates the extent to which each is currently a problem for them using a 5-point scale (0 = not a problem to 4 = serious problem).Scores are summed and transformed to a total score (0-100), with higher scores indicating greater diabetes distress and scores of ≥40 considered as elevated diabetes distress.
The Hypoglycaemia Fear Survey-Short Form (HFS-SF) 18 has 11 items measuring experience of hypoglycaemia-related concerns and behaviours to prevent hypoglycaemia over the past 3 months.Items form three subscales ('Worry', 'Avoidance' and 'Maintain glucose high').Items are rated using a 5-point scale (0 = never to 4 = almost always).For each subscale, items are summed to generate total subscale scores, with higher scores indicating greater fear of hypoglycaemia (i.e.greater concerns or more frequent preventative/avoidance behaviours).
The Gold score 19 is a single question asking: 'Do you know when your hypos are commencing?' with responses on a 7-point scale (1 = always aware and 7 = never aware).Scores of ≥4 indicate impaired awareness of hypoglycaemia (IAH).
The Prospective and Retrospective Memory Questionnaire (PRMQ) 20 includes 16 items assessing a person's frequency of everyday memory errors, rated on a 5-point scale (1 = never to 5 = very often).The items content reflects various aspects of memory, that is, prospective/ retrospective memory, short−/long-term memory, self/ environmental cues and potential errors.There are two scales (prospective and retrospective memory functioning), each with eight items.Item scores for each scale are summed to form composite scores (range: 1-40).Higher scores indicate more (prospective or retrospective) memory errors.
Clinical and demographic data were collated by the NHMRC Clinical Trial Centre (Sydney) (e.g.HbA1c, hypoglycaemia, CGM assessed percentage of Time in Range (TIR) (3.9-10.0mmol/mol; 70-180 mg/dL) or self reported directly by participants through the survey (e.g.age, education).

| Data analysis
All analyses were performed using R 3.5.2,R Studio 1.2.5033 and Stata/SE 16.Descriptive statistics were calculated for demographic and baseline clinical characteristics.
To explore the DME-Q's psychometric properties, pre-randomization data were used.Inter-item Spearman's rho correlations (r s >0.7 considered strong; 0.3-0.7 moderate; <0.3 weak), Bartlett's Test of Sphericity, the Determinant Score and Kaiser-Meyer-Olkin (KMO) statistics were used to examine the correlation strength between items, multicollinearity and sampling size adequacy.Principal components analysis (PCA) with direct oblimin rotation was performed to assess structural validity.Kaiser's eigenvalue (≥1) and proportion of variance explained for each factor were used to inform the number of factors retained.Factor loadings of ≥0.3 were considered meaningful. 21Items with double factor loadings were assigned to a single factor based on loading strength and face validity.A forced one-factor PCA was conducted to examine the validity of combining item scores into a single composite score.Internal consistency reliability of the factors was assessed using Cronbach's alpha, where α ≥ 0.7 was considered satisfactory and α ≥ 0.95 indicated item redundancy. 22onstruct validity was assessed with a single overview item ('Overall, my current method of managing my diabetes suits me well').Convergent and divergent validity assess the extent to which a measure that is theoretically expected to be related or unrelated to another measure is found to be so.They were assessed using Spearman's rho, against measures chosen a priori: The DTSQ 10 and PAID 17 were selected for convergent validity as they measure diabetes-specific satisfaction and distress respectively; divergent validity was assessed with a measure of perceived memory, the generic PRMQ. 20o explore the DME-Q's responsiveness and associations with psychological and clinical factors at 26-week follow-up, the complete trial data set was used.Analysis of covariance (ANCOVA) assessed the effect of the intervention on DME-Q scores at 26-week follow-up, adjusted for pre-randomization DME-Q scores.To investigate whether factors other than intervention affected DME-Q scores, midpoint (13-week) variables were chosen a priori: clinical factors included HbA1c, percentage of time spent in range (% TIR: 3.9-10.0mmol/L); psychological factors included diabetes distress, fear of hypoglycaemia and hypoglycaemia awareness.The relationship between DME-Q and these a priori selected variables was examined using linear regression, adjusting for all confounders pre-randomization including age, diabetes duration, pre-randomization treatment (MDI, insulin pump), intervention arm and severe hypoglycaemia in the past 12 months.Model residuals were assessed visually for non-linearity with appropriate adjustments made if necessary.Interaction terms were included where significance was found, and collinearity was assessed in all models.Results were reported as effect sizes with 95% confidence intervals.Confidence intervals give a range of certainty on the effect size estimate and confidence intervals not overlapping the null value of 0 were considered as a meaningful association.Strength of association was determined by the effect size estimate as well as the width of the confidence bands.

| Participant characteristics
The pre-randomization survey was completed by 149 adults with type 1 diabetes, but n = 29 adults were ineligible for the trial.The remaining 120 were randomized to intervention (n = 61) or standard therapy (n = 59).Participants' characteristics are detailed in Table 1.Mean age was 44 ± 12 years, 52% were women.HbA1c was 61 ± 11 mmol/ mol (7.8 ± 1.0%), diabetes duration was 24 ± 12 years and 47% used an insulin pump prior to the trial.

| Response pattern
All response options were used for every item, except item 10 ('…gives me too much information'), where no participant answered 'strongly agree'.Most items had a negatively skewed distribution.No ceiling effects were observed for any item, while only item 10 showed floor effects.
The three-factor PCA solution accounted for 46% of the total variance.Item 7 ('…reassures me') loaded on both the 'Convenience' and 'Effectiveness' factors (Table 2).Based on factor loading and item content, the item was assigned to the 'Effectiveness' domain.Internal consistency reliability for the 'Convenience', 'Effectiveness' and 'Intrusiveness' subscales was satisfactory ( = 0.74-0.84).Removal of any single item did not affect the reliability of any subscale.
The forced one-factor solution accounted for 28% of the total variance, with factor loadings >0.3 for all items except item 10 (loading = 0.26) (Table 2).Internal consistency reliability for the 'Total satisfaction' scale was strong ( = 0.85).As with the three-factor solution, the removal of any single item from the 'Total satisfaction' scale did not reduce the reliability ( = 0.86).No items were removed as the improvement in reliability was negligible and reduced face validity.

| Convergent and divergent validity
Correlations between subscales were moderate while correlations between each subscale and 'Total satisfaction' were strong (Table 2).Correlations with the single overview item were moderate for the 'Total satisfaction' scale and the three subscales.
There was a moderate positive correlation between the DME-Q 'Total satisfaction' and the DTSQ 'Total satisfaction' (r s = 0.58).There was also a moderate negative correlation (r s = −0.57) between diabetes distress (PAID total) and DME-Q 'Total satisfaction'.As correlations were r s >0.3, convergent validity was supported.Correlations with the PRMQ were all r s <0.3, suggesting appropriate divergent validity between satisfaction and self reported prospective and retrospective memory (Supplementary Table S1).

| Responsiveness
Table 3 shows the mean and 95% confidence intervals of the DME-Q 'Total satisfaction' score and the three subscale scores at 26-week follow-up by trial arm, and the mean difference.The model suggests a meaningful between-group difference in 'Total satisfaction' of 0.4 points, and a difference of nearly a full point in the 'Effectiveness' subscale, both favouring intervention.

| DME-Q and demographic characteristics
Men scored 0.21 points higher, on average, than women on the 'Total satisfaction' scale.Composite scores increased by 0.01 per year increase of age and a 0.007 point increase per year increase of diabetes duration.These associations were similar for the DME-Q subscales.DME-Q composite and subscale scores did not correlate with HbA1c, % TIR (3.9-10.0mmol/mol), for baseline MDI or insulin pump treatment.

| Associations with other psychological/clinical factors
Associations between DME-Q scores at 26-week follow-up and psychological/clinical factors at midpoint (13 weeks) are shown in Table 4. Diabetes distress (PAID) and fear of hypoglycaemia (HFS-SF Worry) were associated with all DME-Q scores; negative associations with the 'Total', 'Convenience' and 'Effectiveness' scores and positive associations with 'Intrusiveness' score.The HFS-SF maintain glucose high subscale was associated with all DME-Q scores, except 'Effectiveness'; negative associations with the 'Total' and 'Convenience' scores, and a positive association with the 'Intrusiveness' score.The HFS-SF Avoidance subscale was associated negatively with 'Total   12) 60 (10)   Severe hypoglycaemia in past 12 months (n = 120) 14 ( 12) 8 ( 13) 6 (10)   Time in range at baseline (3.9-10.0mmol/L), % 54.Note: Results are n (%) or mean (SD).For all questionnaires, higher scores mean higher endorsement of the assessed psychological concept.

Variables
T A B L E 2 DME-Q pre-randomization data: forced three-factor and one-factor structure, variance, internal consistency reliability, interscale correlations and mean (SD) according to trial arm

Convenience Effectiveness Intrusiveness
Total a Scores ranging from 1 to 5, with higher scores indicting greater perceived convenience, effectiveness, or intrusiveness.
satisfaction' only.Awareness of hypoglycaemic symptoms score) predicted a difference in DME-Q total and subscale scores.HbA1c and % TIR were not associated with the DME-Q total or subscale scores.In terms of effect size estimates, no estimate was large compared to their distributions.

| DISCUSSION
The DME-Q, adapted from the validated GME-Q, 8 is designed to capture experiences and satisfaction with diabetes management, irrespective of treatment modalities.This psychometric validation demonstrates that the DME-Q is acceptable, reliable and has satisfactory structural validity.The three-factor and one-factor solutions demonstrate that three subscale scores can be calculated, that is, 'Effectiveness', 'Intrusiveness' and 'Convenience', in addition to a 'Total satisfaction' composite score.Cronbach's alpha coefficients for the total scale and subscales show good reliability with no redundancy.The three subscales correlated moderately with each other, and each had strong correlations with the 'Total satisfaction' score.Together, these findings support the underlying conceptual model and the distinct contribution of all three constructs to the total scale.
The single overview item 'Overall, my current method of managing my diabetes suits me well' correlated moderately with the three subscales.This suggests that the single overview item is not an ideal short form of a 22item measure.For scales that represent narrow concepts, and where a single item focuses on that narrow concept, items will tend to be more highly correlated.This is observed where the score for item 3 ('Convenience') has a 0.91 loading on the factor labelled 'Convenience'.The fact that 'suits me well' does not achieve a high correlation with any subscale or the total scale score is an artefact of the breadth of the three latent psychological constructs (i.e.'Convenience', 'Effectiveness', 'Intrusiveness'), which achieve moderate inter-scale correlations.This remains a limitation for research purposes, and further analysis is needed to examine sensitivity and Note: Mean difference examined using ANCOVA.
specificity of the single overview item.However, may still be useful in clinical settings where a single-item assessment of satisfaction with diabetes management is preferable to no assessment at all.For example, its inclusion may prompt a conversation about the reasons for the appraisal offered by the person and what changes to the regimen may be possible to improve satisfaction/ perceived suitability.Such brief assessments are useful in clinical practice, as shown by use of the Gold score and the DDS-2 in the Type 1 Consultation tool (T1CT), 23 while the DDS-2's correlation with the full 17-item version of the Diabetes Distress Scale is moderate-to-strong (r = 0.69) but not perfect. 24onvergent validity of the DME-Q 'Total satisfaction' was established by the significant positive correlation with the DTSQ (diabetes treatment satisfaction) and negative correlation with the PAID (diabetes distress).These correlations were strong enough to demonstrate satisfactory convergent validity but not so strong as to devalue the unique contribution of the DME-Q in assessing satisfaction with diabetes management.The weak correlation with the PRMQ supports appropriate divergent validity.
At the end of the 26-week trial, participants in the HCL arm reported greater 'Total satisfaction' with their diabetes management compared to the standard therapy group who continued with their usual treatment.The main contributor to improved 'Total satisfaction' was greater 'Effectiveness'.There was no between-group difference in 'Convenience' or 'Intrusiveness'.Greater perceived effectiveness may reflect the participants' clinical outcomes, as an increase in % TIR was observed from 55% at baseline to 70% at 26 weeks in the HCL arm, while this percentage was unchanged in the standard therapy arm (55% at baseline and 26 weeks). 25The absence of difference for 'Convenience' and 'Intrusiveness' suggests that HCL is not necessarily more convenient or less intrusive than MDI or insulin pump alone.Similarly, use of the DME-Q's parent instrument (the GME-Q) in the HypoCOMPaSS trial showed a trend towards participants being more satisfied, in terms of 'Effectiveness', with real-time CGM than self monitoring of (finger-prick) blood glucose, but no difference in 'Convenience' or 'Intrusiveness'. 26e have previously shown that there was no difference between trial arms for diabetes treatment satisfaction, 25 as assessed with the DTSQ. 10 The discrepancy between these findings provides support for the detailed content of the DME-Q, suggesting it may have greater sensitivity for capturing the participants' experiences in complex technology trials.
There were statistically significant associations between diabetes distress (PAID), fear of hypoglycaemia (HFS-SF Worries subscale) and awareness of hypoglycaemia (Gold) mid-trial (13 week) and DME-Q scores at trial end (26-week follow-up) for all subscales and 'Total satisfaction'.Maintaining glucose levels high to cope with fear of hypoglycaemia (HFS-SF Maintain glucose high subscale) was associated with DME-Q 'Convenience', 'Intrusiveness' and 'Total satisfaction' scores, but not with 'Effectiveness'.However, it is unlikely that these associations would be clinically meaningful.Mid-trial HbA1c and % TIR were not associated with any of the 26-week follow-up DME-Q scores.This suggests that perceptions of 'Effectiveness' of HCL at trial end were consolidated by maintenance of glycaemic improvements observed in the latter half of the trial.
This study established the basic psychometric properties of the DME-Q (i.e.structural and construct validity and internal consistency reliability), it also demonstrated the DME-Q's responsiveness to intervention.Other properties need to be examined in the future, such as testretest reliability and the predictive validity of the measure.
The study is further limited by the homogeneity of the sample (mostly English-speaking adults with longstanding type 1 diabetes, higher education, with access to tertiary care, greater insulin pump use compared to overall use in Australians with type 1 diabetes [47% vs. 10%]). 27articipants completed the DME-Q as part of the RCT pre-randomization assessment, thus were willing to take part in the trial and to use advanced technology.These factors limit generalizability and the findings need to be replicated in more diverse populations.In addition, technologies are advancing rapidly, becoming part of routine diabetes management.Satisfaction demonstrated here may be specific to these technologies or to the context of a clinical trial.Therefore, it would be beneficial to use the DME-Q in populations using other treatment and technologies and in real-world settings.

| CONCLUSION
The DME-Q is an acceptable, valid, reliable and responsive measure of satisfaction with diabetes management for adults with type 1 diabetes.Given the importance of satisfaction for continuation with a particular diabetes management approach, the DME-Q has real-world value, and is a useful, brief measure for inclusion in diabetes research studies and clinical settings evaluating new treatments and technologies.

AUTHOR CONTRIBUTIONS
DNO led the overall trial, which was co-designed by AJJ, TWJ, SAM, VS, DNO, ST and JS, and refined with input from LAB, MGB, NDC, PGC, DJH-W, SNS, CH and JAH.Participant screening and enrolment, collection of informed consent and provision of medical care were led by

T A B L E 1
Participant characteristics pre-randomization (N = 149) and post-randomization, by trial arm (n = 120).

T A B L E 4 4 )
Adjusted associations between DME-Q scores at 26-week follow-up and clinical/psychological factors at 13 weeks.Time in range, % (3.9-10.0mmol/mol) 0.007 (−0.004 to 0.018) 0.004 (−0.009 to 0.02) 0.01 (−0.002 to 0.03) −0.005 (−0.02 to 0.009) Diabetes distress (PAID) −0.02 (−0.03 to − 0.01) −0.02 (−0.03 to − 0.01) −0.01 (−0.03 to − 0Cells show DME-Q mean score difference for categorical variables, and DME-Q score increase per unit increase for continuous variables.Bold results indicate effect size estimates and confidence intervals that do not overlap the null value of 0. a Associations adjusted for pre-randomization DME-Q scores, intervention group, age, diabetes duration, pre-randomization treatment and severe hypoglycaemia.T A B L E 3 DME-Q mean scores and differences by trial arm, at 26-week follow-up, adjusted for pre-randomization scores.between-group difference.