INTRODUCTION

SymTrak is a tool for monitoring clinically actionable symptoms that are relevant to multiple chronic conditions (MCCs). The development of SymTrak1 was important because elderly patients attending primary care clinics often have MCCs,2 concomitant complex health care needs, high health care costs,3 and various somatic, emotional, and cognitive symptoms.4, 5 SymTrak is focused on symptoms that are not disease specific. Symptoms presented in primary care often do not yield a disease-based explanation,6 and yet these symptoms predict health care utilization and costs, quality of life, work disability, and mortality.4, 5, 7,8,9,10,11 For example, sleep disturbance, pain, anxiety, depression, and low energy/fatigue are prevalent, chronic, disabling, and under-treated symptoms that co-occur,12,13,14 interact synergistically, and adversely impact health and treatment response, which suggests clinicians should address multiple symptoms simultaneously.6, 12, 14, 15 A focus on clinically actionable symptoms is supported by the fact that symptoms account for over half of all US outpatient visits annually.10 Furthermore, some interventions are effective for more than one symptom category,6 making it wise to monitor the frequency or severity of symptoms with a tool that can improve diagnosis,16 guide treatment, and monitor treatment response.4 Developing a caregiver-report version was important because up to 57% of elderly patients are accompanied by an informal caregiver during physician visits.17

The development of SymTrak has been previously described.1 Online Appendix 1 shows the item-level similarities and differences in wording and formatting between SymTrak and legacy instruments. The present paper reports findings from a validation study in 600 participants. Internal consistency and test–retest reliability, construct validity, and sensitivity to change from baseline to 3 months were assessed.

METHODS

Setting and Sample

The 600 participants (200 patient–caregiver dyads and 200 non-dyadic patients without an identified caregiver) were recruited from an academic-affiliated primary care network of clinics. Patients with and without caregivers were equally sampled to investigate whether SymTrak was psychometrically sound in both groups. The study was approved by the institutional review board and all participants provided written informed consent. Patient inclusion criteria were the following: (1) age ≥ 65 years, (2) ≥ one primary care visit in the past 12 months, (3) ≥ one chronic condition according to medical records, and (4) for those participants who had an identified informal caregiver, the caregiver had to be ≥ 21 years of age and willing to participate in the study. Patient exclusion criteria were permanent residency at a long-term care facility or presence of a serious mental illness such as bipolar disorder or schizophrenia.

Measures

Participants (N = 600) completed a brief survey by interview consisting of demographics, SymTrak, and the Health Utility Index Mark 3 (HUI3),18 at baseline and 3 months post-baseline. A subsample of 60 patient–caregiver dyads and 60 non-dyadic individual patients also were re-interviewed 24 h after baseline (n = 180). All interviews were telephone-administered by research assistants.

The HUI3 is a general preference-based measure of health-related quality of life (HRQOL) for which the overall utility score of 0 represents death and 1 represents perfect health. A criterion standard for 3-month reliable HRQOL change groups (Decline, Stable, or Improve) was formulated based on ± 1 standard error of measurement (SEM) in the HUI3 overall utility score. The HUI3 SEM was calculated as 1 SD of the change in the HUI3 overall utility score from baseline to 3 months, multiplied by the square root of 1 minus the HUI3 24-h test–retest reliability coefficient. The value of 1 SEM equaled 0.092 for non-dyadic patients, 0.90 for dyadic patients, and 0.085 for caregivers. A value of 0.089 was used for 1 SEM because it was between these values and it was the value for 1 SEM for all patients combined.

SymTrak consists of 23 items focused on clinically actionable symptoms measured on the following scale: 0 = Never, 1 = Sometimes, 2 = Often, 3 = Always. Two general health items (measured as Poor, Fair, Good, Very Good, Excellent) were assessed separately for physical and emotional health, and are included in the tool to provide researchers with validity correlations and to support clinical interpretations because nurses and physicians reported during focus groups in the SymTrak developmental study1 that they appreciated these general health ratings as additional information for gauging the patient’s overall health.

Analysis

Exploratory factor analysis (EFA) was performed using the nonlinear ordered probit link for ordinal categorical items and WLSMV estimation in MPLUS software. The EFA was performed separately for patient- and caregiver-reported data, and results were explored using VARIMAX and PROMAX rotation. Cronbach’s alpha was used to assess internal consistency reliability. The absolute-agreement version of the intra-class correlation coefficient (ICC) was used to assess test–retest reliability, with occasions specified as a random effect. For sensitivity to 3-month change, the mean change and within-group standardized response mean (SRM) effect size (mean change divided by SD of change) were computed for SymTrak total scores, separately for each of three reliable HRQOL change groups (Decline, Stable, or Improve). A general linear model (GLM) was used to compare SymTrak change scores by testing the overall difference between all three HUI3 change groups using the omnibus F test, and then testing all pairwise differences (e.g., Decline vs Improve) using Tukey–Kramer post hoc tests with family-wise type I error controlled at 0.05. Linear regression was used to test the concurrent predictive validity between baseline SymTrak scores and the dependent variable of baseline HUI3 overall utility scores (0–100 metric) while adjusting for the patient’s demographic characteristics. All tests were two-sided using 0.05 for alpha.

RESULTS

Participant Characteristics

Patients and caregivers were diverse with respect to race, education, income, and marital status (Table 1). Patients and caregivers from recruited dyads were allowed to remain in the study if their partner subsequently decided not to participate; thus, the number of caregivers (n = 203) is slightly greater than the number of dyadic patients (n = 200). The majority of caregivers were either a child (43%) or a spouse/partner (36%) of the dyadic patients (Table 1).

Table 1 Participant Characteristics
Full size table

Compared to patients without an identified caregiver, patients with a recruited caregiver were significantly older by an average of 2 years (p = 0.01) but also more likely to be married or living together (p < 0.0001) and with higher household income (p < 0.0001), although income was low for all groups (Table 1). Given these demographic characteristics, it is not surprising that the two patient groups did not differ significantly on HUI3 functional status, SymTrak total symptom scores, or poor-to-excellent general health ratings. African Americans were well represented in this sample, comprising 49% of dyadic patients and caregivers, and 60% of patients without caregivers. In all three samples, the majority of participants reported the patient’s health to be in either fair or good health on both emotional and physical health (Table 1).

Data Distribution

All 23 items on SymTrak received endorsement for all 4 response options by the patients (Table 2). The response option distribution was similar for patients with and without an identified caregiver; therefore, all patients were combined for Table 2. A majority of patients endorsed the Never or Sometimes options for most items; however, for nearly half of the tool (10 items), a meaningful percentage (17 to 52%) of patients endorsed either Often or Always (Table 2). Thus, SymTrak exhibited reasonable spread in its item response scores. Patients reported a relatively higher level of endorsement (indicated by item means of 1.0 or greater) for items assessing fatigue (no. 1), sleep disturbance (no. 2), pain (no. 3–no. 5), and mobility (no. 12). These item statistics were similar when the symptoms of patients were reported by their caregivers (Online Appendix 2). The SymTrak total score has a possible range from 0 to 69 and was approximately normally distributed in all three participant groups (Online Appendix 3). Concordance between dyadic patients and their caregivers was reasonably close for most items, with a similar mean item score and a kappa coefficient representing moderate agreement (Online Appendix 4).

Table 2 Patient-Reported SymTrak Item Distributions and Loadings (N = 400)
Full size table

Factor Analysis

The EFA scree plot revealed a single dominant dimension for patients and caregivers (Fig. 1). Loadings from the one-factor solution were good (i.e., > 0.40) for 20 items, ranging from 0.43 to 0.70; for three items (no. 7, no. 11, and no. 20), loadings were lower but adequate (between 0.30 and 0.40; Table 2). Thus, factor analysis supported the validity of using the total score as an overall summary of symptoms. The validity of additional subscale scoring was investigated by exploring rotated factor loadings for the 2-, 3-, 4-, and 5-factor EFA solutions; however, none of the factor models, regardless of rotation method, offered a meaningfully clearer factor structure (i.e., higher loadings without cross-loadings on other domains) compared to the one-factor model. For example, although cognitive items often clustered together, and emotional items usually clustered together, several physical items demonstrated cross-loadings with the cognitive or emotional domains in all rotated factor solutions.

Fig. 1
figure 1

Factor analysis scree plot.

Full size image

Reliability

Coefficient alpha for the SymTrak total score was high for non-dyadic patients (0.85), dyadic patients (0.86), and caregivers (0.86). Test–retest reliability also revealed high agreement between baseline and 24-h retest values for non-dyadic patients (0.87), dyadic patients (0.88), and caregivers (0.91). For readers interested in the reliability over all patients combined, internal consistency and test–retest reliability were 0.85 and 0.87, respectively.

Construct Validity

SymTrak demonstrated a dose–response relationship with the single-item general health ratings (Online Appendix 5). Specifically, the mean SymTrak total score generally decreased as patient health rating categories increased from Poor to Excellent (the two exceptions occurred in rating categories with less than 10 participants). This dose–response relationship was evident for both physical and emotional general health ratings, and for both patient-reported and caregiver-reported data.

Spearman correlations between the SymTrak total score and the HUI3 overall utility score were strong: − 0.83 for non-dyadic patients, − 0.84 for dyadic patients, (− 0.84 when disseminating for all patients combined), and − 0.69 for caregivers. The scatterplot between SymTrak total score and HUI3 overall utility score revealed a reasonably linear relationship; as symptom burden decreased, preference-based HRQOL increased (Online Appendix 6). Baseline SymTrak total score was strongly associated with baseline HUI3 HRQOL after adjusting for demographic covariates (standardized linear regression beta coefficient [STB] = − 0.84 for each of the patient groups and − 0.70 for caregivers; Table 3). The following is an example of interpreting the models in Table 3. For every 1 SD increase in caregiver-reported SymTrak total score, the patient’s caregiver-reported HUI3 utility score decreased on average by 0.70 SD after adjusting for the patient’s age, sex, race, education, household income, and marital status (model 3). Older age and very low income significantly predicted poorer HRQOL but their impact on HRQOL was relatively small compared to the impact of SymTrak-measured symptoms (model 1, STBage = − 0.09, STBSymTrak = − 0.84; model 3, STB > $30,000 vs < $15,000 = 0.16, STBSymTrak = − 0.70).

Table 3 SymTrak as a Predictor of Health Utility Index Mark 3 (HUI3)
Full size table

Sensitivity to Change

Validity was supported by the sensitivity-to-change analysis (Table 4). The SRM represents the magnitude of the change in the SymTrak total score over 3 months. For all three participant samples, the SRM was reasonably centered near 0, as expected, for patients in the HUI3 Stable group (Table 4).

Table 4 Sensitivity-to-Change Validity of SymTrak
Full size table

In addition, for patient-reported and caregiver-reported scores, the SRM for SymTrak was in the anticipated negative direction for the group of patients that declined in HRQOL and in the positive direction for the group of patients that improved in HRQOL. For all three samples, the omnibus p value for comparing SymTrak change scores between the three HRQOL change groups was significant, and the pairwise difference between HRQOL Decline and Improve groups (D vs I) was significant (Table 4). Pairwise comparisons of the Decline and Improve groups with the reference (Stable) group were not significant, except for the caregiver-reported SymTrak total score which significantly differentiated between Decline and Stable HRQOL groups due to caregivers reporting a larger SymTrak SRM for the Decline group.

DISCUSSION

The SymTrak tool demonstrated an adequate spread of item response scores across its 23 symptom-focused items. The EFA revealed one dominant dimension for explaining the correlation among the SymTrak item scores, with adequately strong factor loadings for the one-factor solution, for both patient-reported and caregiver-reported data. The internal consistency reliability and test–retest reliability coefficients were very high for the SymTrak total score. Perhaps future factor analyses in other samples will reveal valid subscale scoring options despite the fact that EFA in this data, using state-of-the-art nonlinear factor analysis, did not reveal clean subscales. Physical symptoms (e.g., sleep disturbance, fatigue, pain) often cluster with anxiety, depression, and cognitive symptoms in primary care populations. It is possible that these symptoms correlate across domains even more highly in populations with MCCs. The baseline cross-sectional analyses supported the construct validity of the SymTrak total score including dose–response relationships across poor-to-excellent categories of physical and emotional general health ratings. Also shown were high correlations with the HUI3 overall utility score, regardless of whether the patients’ symptoms and HRQOL were reported by the patients themselves or by their informal caregivers. When demographics were included in a linear regression model for predicting the HUI3 overall utility score, the SymTrak total score was much more strongly associated with preference-based HRQOL than were the patient’s demographic variables including age. Importantly, this finding occurred in elderly patients with a wide age range (from 65 to 95) in which 41% were 75 or older.

An important aspect of validity, especially for a tool developed primarily for monitoring treatment response, is sensitivity to change. The SymTrak total score demonstrated good sensitivity to change, especially considering the short 3-month interval. For both patient-reported and caregiver-reported data, the SymTrak total score was able to distinguish between the three HUI3-defined groups of reliable change in HRQOL, most notably between declining and improving HUI3-based HRQOL. However, only caregiver-reported SymTrak scores were sensitive enough to correlate with declining HRQOL and thereby detect the difference between declining and stable HRQOL over a brief 3-month period. This suggests that caregiver-reported SymTrak data may be particularly useful for monitoring patients’ symptoms during times of suspected declines. This should be further investigated in future longitudinal studies in which caregiver- and patient-reported SymTrak scores are compared to each other regarding their sensitivity for earlier detection of clinically verified declines in HRQOL and other measures of functioning.

Limitations

Because the sample only included adults 65 and older, generalizability to individuals less than 65 needs to be investigated in future studies. Second, sensitivity to change of SymTrak should be studied over a longer interval than 3 months and with additional validators besides the HUI3. Also, responsiveness to treatment should be tested in clinical trials and other prospective studies aimed at reducing symptom burden for patients with MCCs. Third, our sample of older adults attending primary care with chronic conditions was generally cognitively intact; therefore, the performance of SymTrak in patients with mild to moderate cognitive impairment should be evaluated. Fourth, all interviews were administered by telephone; therefore, we could not compare with other modes of administration. However, a previous study proved that self-administration of SymTrak in clinics, for patients aged 65 and older and their informal caregivers, is psychometrically sound and efficient for both patient-reported and caregiver-reported scores.1 The use of SymTrak scores (both baseline and change over time) to predict future health care utilization would further inform predictive validity.

Research and Clinical Implications

The SymTrak total score was approximately normally distributed. This is a convenient advantage in parametric statistical models especially when the score is used as a dependent variable. For example, SymTrak may be useful for assessing response to treatment in randomized controlled trials or in observational correlational studies. In populations with MCCs, and where treatment of one symptom synergistically affects response on other symptoms,6, 12, 14, 15 the SymTrak total score may serve as a useful overall measure of treatment outcome in research and clinical practice, especially given its promising sensitivity to detect change in HRQOL over a brief 3-month period. SymTrak performed psychometrically well in all three samples (i.e., both patient groups and caregivers), which represents a strength of the scale for research and clinical practice.