Abstract
The aim of this study is to show the relationship between test-retest reproducibility and responsiveness and to introduce the smallest real difference (SRD) approach, using the sickness impact profile (SIP) in chronic stroke patients as an example. Forty chronic stroke patients were interviewed twice by the same examiner, with a 1-week interval. All patients were interviewed during the qualification period preceding a randomized clinical trial. Test-retest reproducibility has been quantified by the intraclass correlation coefficient (ICC), the standard error of measurement (SEM) and the related smallest real difference (SRD). Responsiveness was defined as the ratio of the clinically relevant change to the SD of the within-stable-subject test-retest differences. The ICC for the total SIP was 0.92, whereas the ICCs for the specified SIP categories varied from 0.63 for the category ‘recreation and pastime’ to 0.88 for the category ‘work’. However, both the SEM and the SRD far more capture the essence of the reproducibility of a measurement instrument. For instance, a total SIP score of an individual patient of 28.3% (which is taken as an example, being the mean score in the study population) should decrease by at least 9.26% or approximately 13 items, before any improvement beyond reproducibility noise can be detected. The responsiveness to change of a health status measurement instrument is closely related to its test-retest reproducibility. This relationship becomes more evident when the SEM and the SRD are used to quantify reproducibility, than when ICC or other correlation coefficients are used.
Similar content being viewed by others
References
Hopkins WG. Measures of reliability in sports medicine and science. Sports Med 2000; 30: 1–15.
Streiner DL, Norman GR. Health Measurement Scales. A Practical Guide to Their Development and Use. Oxford: Oxford University Press, 1993.
Guyatt G, Walter S, Norman G. Measuring change over time: Assessing the usefulness of evaluative instruments. J Chron Dis 1987; 40: 171–178.
Bennekom CAM van, Jelles F, Lankhorst GJ, Bouter LM. Responsiveness of the rehabilitation activities profile and the Barthel index. J Clin Epidemiol 1996; 49: 39–44.
Moertel CG, Hanley JA. The effect of measuring error on the results of therapeutic trials in advanced cancer. Cancer 1976; 38: 388–394.
Beckerman H, Becher J, Lankhorst GJ, Verbeek ALM. Walking ability of stroke patients: Efficacy of tibial nerve blocking and a polypropylene ankle-foot orthosis. Arch Phys Med Rehabil 1996; 77: 1144–1151.
Gilson BS, Gilson JS, Bergner M, et al. The sickness impact profile. Development of an outcome measure of health care. Am J Public Health 1975; 65: 1304–1310.
Bergner M, Bobbitt RA, Kressel S, Pollard WE, Gilson BS, Morris JR. The sickness impact profile: Conceptual formulation and methodology for the development of a health status measure. Int J Health Serv 1976; 6: 393–415.
Bergner M, Bobbitt RA, Pollard WE, Martin DP, Gilson BS. The sickness impact profile: Validation of a health status measure. Med Care 1976; 14: 57–67.
McMahon LP, Dawborn JK. Subjective quality of life assessment in hemodialysis patients at different levels of hemoglobin following use of recombinant human erythropoietin. Am J Nephrol 1992; 12: 162–169.
Hazard Munro B, Creamer AM, Haggerty MR, Cooper FS. Effect of relaxation therapy on post-myocardial infarction patients' rehabilitation. Nurs Res 1988; 37: 231–235.
Hyde E. Acupressure therapy for morning sickness. A controlled clinical trial. J Nurse-Midwifery 1989; 34: 171–178.
Pollard WE, Bobbitt RA, Bergner M, Martin DP, Gilson BS. The sickness impact profile: Reliability of a health status measure. Med Care 1976; 14: 146–155.
Bergner M, Bobbitt RA, Carter WB, Gilson BS. The sickness impact profile: Development and final revision of a health status measure. Med Care 1981; 19: 787–805.
Bruin AF de, Witte LP de, Stevens F, Diederiks JPM. Sickness impact profile: The state of the art of a generic functional status measure. Soc Sci Med 1992; 35: 1003–1014.
Deyo RA, Diehl AK. Measuring physical and psychosocial function in patients with low back pain. Spine 1983; 8: 635–642.
Deyo RA. Comparative validity of the sickness impact profile and shorter scales for functional assessment in low-back pain. Spine 1986; 11: 951–954.
Jensen MP, Strom SE, Turner JA, Romano JM. Validity of the sickness impact profile Roland scale as a measure of dysfunction in chronic pain patients. Pain 1992; 50: 157–162.
Sullivan M, Ahlmen M, Archenholtz B, Svensson G. Measuring health in rheumatic disorders by means of a Swedish version of the sickness impact profile. Scand J Rheumatol 1986; 15: 193–200.
Jacobs HM, Touw-Otten FWMM, Melker RA de. The evaluation of changes in functional health status in patients with abdominal complaints. J Clin Epidemiol 1996; 49: 163–171.
Damiano AM, Patrick DL, Guzman GI, et al. Measurement of health-related quality of life in patients with amyotrophic lateral sclerosis in clinical trials of new therapies. Med Care 1999; 37: 15–26.
Visser MC, Koudstaal PJ, Erdmann RA, et al. Measuring quality of life in patients with myocardial infarction or stroke: A feasibility study of four questionnaires in The Netherlands. J Epidemiol Commun Health 1995; 49: 513–517.
Witte L de, Jacobs H, Horst F van der, Luttik A, Joosten J, Philipsen H. De waarde van de sickness impact profile als maat voor het functioneren van patiënten. Gezondheid en Samenleving 1987; 8: 120–127.
Jacobs HM, Luttik A, Touw-Otten FWMM, Kastein M, De Melker RA. Measuring impact of sickness in patients with nonspecific abdominal complaints in a Dutch family practice setting. Med Care 1992; 30: 244–251.
Arendzen JH, Duijn H van, Beckmann MKF, Harlaar J, Vogelaar TW, Prevo AJH. Diagnostic blocks of the tibial nerve in spastic hemiparesis. Effects on clinical, electrophysiological and gait parameters. Scand J Rehab Med 1992; 24: 75–81.
Shavelson RJ, Webb NM. Generalizability Theory: A Primer. Newbury Park: Sage Publications, 1991.
Carter WB, Bobbitt RA, Bergner M, Gilson BS. Validation of an interval scaling: The sickness impact profile. Health Services Res 1976; 517–528.
Roebroeck ME, Harlaar J, Lankhorst GJ. The application of the generalizability theory to reliability assessment: An illustration using isometric force measurement. Phys Ther 1993; 73: 386–401.
Deyo RA, Diehr P, Patrick DL. Reproducibility and responsiveness of health status measures. Statistics and strategies for evaluation. Controlled Clinic Trials 1991; 12: 142S–158S.
Stratford PW. Reliability: Consistency or differentiating among subjects? Phys Ther 1989; 69: 299–300.
Guyatt GH, Kirschner B, Jaeschke R. Measuring health status: What are the necessary measurement properties? J Clin Epidemiol 1992; 45: 1341–1345.
Pfennings LEMA, Ploeg HM van der, Cohen L, Polman CH. A comparison of responsiveness indices in multiple sclerosis patients. Qual Life Res 1999; 8: 481–489.
Brennan RL. Elements of Generalizability Theory. Iowa city: ACT publications, 1983.
Crick JE, Brennan RL. Manual for GENOVA: A Generalized Analysis of Variance System. Iowa city: American College Testing Program, 1983.
Kramer MS, Feinstein AR. Clinical biostatistics. LIV. The biostatistics of concordance. Clin Pharmacol Ther 1981; 29: 111–123.
Bartko JJ. On various intraclass correlation reliability coefficients. Psycholog Bull 1976a; 83: 762–765.
Bartko JJ, Carpenter WT. On the methods and theory of reliability. J Nerv Mental Dis 1976; 163: 307–317.
Rankin G, Stokes M. Reliability of assessment tools in rehabilitation: An illustration of appropriate statistical analyses. Clinic Rehabil 1998; 12: 187–199.
Lachs MS. The more things change.... J Clin Epidemiol 1993; 46: 1091–1092.
Hébert R, Spiegelhalter DJ, Brayne C. Setting the minimal metrically detectable change on disability rating scales. Arch Phys Med Rehabil 1997; 78: 1305–1308.
Schuling J, Greidanus J, Meyboom-de Jong B. Measuring functional status of stroke patients with the sickness impact profile. Disabil Rehabil 1993; 15: 19–23.
Stratford P, Solomon P, Binkley J, Finch E, Gill C. Sensitivity of sickness impact profile items to measure change over time in a low back pain patient group. Spine 1993; 18: 1723–1727.
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Beckerman, H., Roebroeck, M., Lankhorst, G. et al. Smallest real difference, a link between reproducibility and responsiveness. Qual Life Res 10, 571–578 (2001). https://doi.org/10.1023/A:1013138911638
Issue Date:
DOI: https://doi.org/10.1023/A:1013138911638