Testing item response theory invariance of the standardized Quality-of-life Disease Impact Scale (QDIS^®) in acute coronary syndrome patients: differential functioning of items and test

Abstract

Purpose

The Quality-of-life (QOL) Disease Impact Scale (QDIS^®) standardizes the content and scoring of QOL impact attributed to different diseases using item response theory (IRT). This study examined the IRT invariance of the QDIS-standardized IRT parameters in an independent sample.

Method

The differential functioning of items and test (DFIT) of a static short-form (QDIS-7) was examined across two independent sources: patients hospitalized for acute coronary syndrome (ACS) in the TRACE-CORE study (N = 1,544) and chronically ill US adults in the QDIS standardization sample. “ACS-specific” IRT item parameters were calibrated and linearly transformed to compare to “standardized” IRT item parameters. Differences in IRT model-expected item, scale and theta scores were examined. The DFIT results were also compared in a standard logistic regression differential item functioning analysis.

Results

Item parameters estimated in the ACS sample showed lower discrimination parameters than the standardized discrimination parameters, but only small differences were found for thresholds parameters. In DFIT, results on the non-compensatory differential item functioning index (range 0.005–0.074) were all below the threshold of 0.096. Item differences were further canceled out at the scale level. IRT-based theta scores for ACS patients using standardized and ACS-specific item parameters were highly correlated (r = 0.995, root-mean-square difference = 0.09). Using standardized item parameters, ACS patients scored one-half standard deviation higher (indicating greater QOL impact) compared to chronically ill adults in the standardization sample.

Conclusion

The study showed sufficient IRT invariance to warrant the use of standardized IRT scoring of QDIS-7 for studies comparing the QOL impact attributed to acute coronary disease and other chronic conditions.

Appendices

Appendix 1: illustration of calculations

Item response theory indeterminacy

The logit function in an IRT model is defined by

$$Logit \, \left[ {P_{i} \left( {\theta_{j} } \right)} \right] \, = \, a_{i} (\theta_{j} {-} \, b_{i)}$$

where P _i (θ _j ) is the probability of endorsing item response category i for person j, a _i and b _i are the item discrimination and threshold parameters for response category i, respectively, and θ _j is the IRT-based theta score for person j. The logit will preserve the same value if a _i , θ _j , and b_i were replaced by $a_{i}^{'}$, $\theta_{j}^{'}$, and $b_{i}^{'}$, respectively, which follow a set of linear transformations of

$$b_{i}^{'} = \, A* \, b_{i} + \, B$$

$$\theta_{j}^{'} = \, A*\theta_{j} + \, B$$

$$a_{i}^{'} = \, {{a_{i} } \mathord{\left/ {\vphantom {{a_{i} } A}} \right. \kern-0pt} A}$$

where A is the slope and B is the intercept of the linear transformations.

Slope (A) and intercept (B) of linear transformation using IRT theta scores

$$A \, = \, {{\sigma \, \left( {\theta_{\text{ST}} } \right)} \mathord{\left/ {\vphantom {{\sigma \, \left( {\theta_{\text{ST}} } \right)} {\sigma \, \left( {\theta_{\text{ACS}} } \right)}}} \right. \kern-0pt} {\sigma \, \left( {\theta_{\text{ACS}} } \right)}}$$

$$B \, = \, \mu \left( {\theta_{\text{ST}} } \right){-}A*\mu \left( {\theta_{\text{ACS}} } \right)$$

where σ (θ _ST) and σ (θ _ACS) are the SD of the IRT scores of ACS patients using the standardized (θ _ST) and the ACS-specific (θ _ACS) IRT item parameters, respectively. μ (θ _ST) and μ (θ _ACS) are the means of the IRT scores.

Non-compensatory differential item functioning (NCDIF)

$${\text{NCDIF}}_{i} = \frac{{\mathop \sum \nolimits_{j = 1}^{N} \left( {S_{{{\text{ST}},i,j}} - S_{{{\text{ACS,}}i,j}} } \right)^{2} }}{N}$$

where S _ST,i,j is the IRT model-expected item response score of item i for respondent j using the standardized IRT item parameters. S _ACS,i,j is the IRT model-expected item response score of item i for respondent j using the ACS-LT IRT item parameters. N is the sample size of ACS patients.

Test differential functioning (DTF)

$${\text{DTF}} = \frac{{\mathop \sum \nolimits_{j = 1}^{N} \left( {{\text{TS}}_{{{\text{ST}},j}} - {\text{TS}}_{{{\text{ACS}},j}} } \right)^{2} }}{N}$$

where TS_ST,j is the IRT model-expected scale(test) score for respondent j using the standardized IRT item parameters. TS_ACS,j is the IRT model-expected scale(test) score for respondent j using the ACS-LT IRT item parameters. N is the sample size of ACS patients.

Compensatory differential item functioning (CDIF)

$$d_{i} (j) = S_{{{\text{ST}},i,j }} {-}S_{{{\text{ACS}},i,j}}$$

$$D(j) = {\text{TS}}_{{{\text{ST}},j}} {-}{\text{TS}}_{{{\text{ACS}},j}}$$

$${\text{CDIF}}_{i} = {\text{ COV}}\left( {d_{i} , \, D} \right) + \, \mu \, \left( {d_{i} } \right)*\mu \left( D \right)$$

where d _i is the difference in the IRT model-expected item response score of item i between the standardized and the ACS-LT IRT item parameters. D is the difference in IRT model-expected scale score between the standardized and the ACS-LT IRT item parameters. COV(d _i , D) is the covariance of d _i and D. μ (d _i ) and μ (D) are their means, respectively. Of note that DTF is equivalent to the sum of CDIF _i added across all the items: ${\text{DTF}} = \mathop \sum \nolimits_{i = 1}^{I} {\text{CDIF}}_{i}$.

Appendix 2

Raw residual plots of fitting the IRT model in the ACS patients for the QDIS-7 (ordered by row from Item 1 to Item 7)