Skip to main content
Log in

Smallest real difference, a link between reproducibility and responsiveness

Quality of Life Research Aims and scope Submit manuscript

Abstract

The aim of this study is to show the relationship between test-retest reproducibility and responsiveness and to introduce the smallest real difference (SRD) approach, using the sickness impact profile (SIP) in chronic stroke patients as an example. Forty chronic stroke patients were interviewed twice by the same examiner, with a 1-week interval. All patients were interviewed during the qualification period preceding a randomized clinical trial. Test-retest reproducibility has been quantified by the intraclass correlation coefficient (ICC), the standard error of measurement (SEM) and the related smallest real difference (SRD). Responsiveness was defined as the ratio of the clinically relevant change to the SD of the within-stable-subject test-retest differences. The ICC for the total SIP was 0.92, whereas the ICCs for the specified SIP categories varied from 0.63 for the category ‘recreation and pastime’ to 0.88 for the category ‘work’. However, both the SEM and the SRD far more capture the essence of the reproducibility of a measurement instrument. For instance, a total SIP score of an individual patient of 28.3% (which is taken as an example, being the mean score in the study population) should decrease by at least 9.26% or approximately 13 items, before any improvement beyond reproducibility noise can be detected. The responsiveness to change of a health status measurement instrument is closely related to its test-retest reproducibility. This relationship becomes more evident when the SEM and the SRD are used to quantify reproducibility, than when ICC or other correlation coefficients are used.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Hopkins WG. Measures of reliability in sports medicine and science. Sports Med 2000; 30: 1–15.

    Article  PubMed  CAS  Google Scholar 

  2. Streiner DL, Norman GR. Health Measurement Scales. A Practical Guide to Their Development and Use. Oxford: Oxford University Press, 1993.

    Google Scholar 

  3. Guyatt G, Walter S, Norman G. Measuring change over time: Assessing the usefulness of evaluative instruments. J Chron Dis 1987; 40: 171–178.

    Article  PubMed  CAS  Google Scholar 

  4. Bennekom CAM van, Jelles F, Lankhorst GJ, Bouter LM. Responsiveness of the rehabilitation activities profile and the Barthel index. J Clin Epidemiol 1996; 49: 39–44.

    Article  PubMed  CAS  Google Scholar 

  5. Moertel CG, Hanley JA. The effect of measuring error on the results of therapeutic trials in advanced cancer. Cancer 1976; 38: 388–394.

    Article  PubMed  CAS  Google Scholar 

  6. Beckerman H, Becher J, Lankhorst GJ, Verbeek ALM. Walking ability of stroke patients: Efficacy of tibial nerve blocking and a polypropylene ankle-foot orthosis. Arch Phys Med Rehabil 1996; 77: 1144–1151.

    Article  PubMed  CAS  Google Scholar 

  7. Gilson BS, Gilson JS, Bergner M, et al. The sickness impact profile. Development of an outcome measure of health care. Am J Public Health 1975; 65: 1304–1310.

    Article  PubMed  CAS  Google Scholar 

  8. Bergner M, Bobbitt RA, Kressel S, Pollard WE, Gilson BS, Morris JR. The sickness impact profile: Conceptual formulation and methodology for the development of a health status measure. Int J Health Serv 1976; 6: 393–415.

    PubMed  CAS  Google Scholar 

  9. Bergner M, Bobbitt RA, Pollard WE, Martin DP, Gilson BS. The sickness impact profile: Validation of a health status measure. Med Care 1976; 14: 57–67.

    PubMed  CAS  Google Scholar 

  10. McMahon LP, Dawborn JK. Subjective quality of life assessment in hemodialysis patients at different levels of hemoglobin following use of recombinant human erythropoietin. Am J Nephrol 1992; 12: 162–169.

    Article  PubMed  CAS  Google Scholar 

  11. Hazard Munro B, Creamer AM, Haggerty MR, Cooper FS. Effect of relaxation therapy on post-myocardial infarction patients' rehabilitation. Nurs Res 1988; 37: 231–235.

    Google Scholar 

  12. Hyde E. Acupressure therapy for morning sickness. A controlled clinical trial. J Nurse-Midwifery 1989; 34: 171–178.

    Article  PubMed  CAS  Google Scholar 

  13. Pollard WE, Bobbitt RA, Bergner M, Martin DP, Gilson BS. The sickness impact profile: Reliability of a health status measure. Med Care 1976; 14: 146–155.

    PubMed  CAS  Google Scholar 

  14. Bergner M, Bobbitt RA, Carter WB, Gilson BS. The sickness impact profile: Development and final revision of a health status measure. Med Care 1981; 19: 787–805.

    PubMed  CAS  Google Scholar 

  15. Bruin AF de, Witte LP de, Stevens F, Diederiks JPM. Sickness impact profile: The state of the art of a generic functional status measure. Soc Sci Med 1992; 35: 1003–1014.

    Article  PubMed  CAS  Google Scholar 

  16. Deyo RA, Diehl AK. Measuring physical and psychosocial function in patients with low back pain. Spine 1983; 8: 635–642.

    PubMed  CAS  Google Scholar 

  17. Deyo RA. Comparative validity of the sickness impact profile and shorter scales for functional assessment in low-back pain. Spine 1986; 11: 951–954.

    PubMed  CAS  Google Scholar 

  18. Jensen MP, Strom SE, Turner JA, Romano JM. Validity of the sickness impact profile Roland scale as a measure of dysfunction in chronic pain patients. Pain 1992; 50: 157–162.

    Article  PubMed  CAS  Google Scholar 

  19. Sullivan M, Ahlmen M, Archenholtz B, Svensson G. Measuring health in rheumatic disorders by means of a Swedish version of the sickness impact profile. Scand J Rheumatol 1986; 15: 193–200.

    PubMed  CAS  Google Scholar 

  20. Jacobs HM, Touw-Otten FWMM, Melker RA de. The evaluation of changes in functional health status in patients with abdominal complaints. J Clin Epidemiol 1996; 49: 163–171.

    Article  PubMed  CAS  Google Scholar 

  21. Damiano AM, Patrick DL, Guzman GI, et al. Measurement of health-related quality of life in patients with amyotrophic lateral sclerosis in clinical trials of new therapies. Med Care 1999; 37: 15–26.

    Article  PubMed  CAS  Google Scholar 

  22. Visser MC, Koudstaal PJ, Erdmann RA, et al. Measuring quality of life in patients with myocardial infarction or stroke: A feasibility study of four questionnaires in The Netherlands. J Epidemiol Commun Health 1995; 49: 513–517.

    CAS  Google Scholar 

  23. Witte L de, Jacobs H, Horst F van der, Luttik A, Joosten J, Philipsen H. De waarde van de sickness impact profile als maat voor het functioneren van patiënten. Gezondheid en Samenleving 1987; 8: 120–127.

    Google Scholar 

  24. Jacobs HM, Luttik A, Touw-Otten FWMM, Kastein M, De Melker RA. Measuring impact of sickness in patients with nonspecific abdominal complaints in a Dutch family practice setting. Med Care 1992; 30: 244–251.

    PubMed  CAS  Google Scholar 

  25. Arendzen JH, Duijn H van, Beckmann MKF, Harlaar J, Vogelaar TW, Prevo AJH. Diagnostic blocks of the tibial nerve in spastic hemiparesis. Effects on clinical, electrophysiological and gait parameters. Scand J Rehab Med 1992; 24: 75–81.

    CAS  Google Scholar 

  26. Shavelson RJ, Webb NM. Generalizability Theory: A Primer. Newbury Park: Sage Publications, 1991.

    Google Scholar 

  27. Carter WB, Bobbitt RA, Bergner M, Gilson BS. Validation of an interval scaling: The sickness impact profile. Health Services Res 1976; 517–528.

  28. Roebroeck ME, Harlaar J, Lankhorst GJ. The application of the generalizability theory to reliability assessment: An illustration using isometric force measurement. Phys Ther 1993; 73: 386–401.

    PubMed  CAS  Google Scholar 

  29. Deyo RA, Diehr P, Patrick DL. Reproducibility and responsiveness of health status measures. Statistics and strategies for evaluation. Controlled Clinic Trials 1991; 12: 142S–158S.

    CAS  Google Scholar 

  30. Stratford PW. Reliability: Consistency or differentiating among subjects? Phys Ther 1989; 69: 299–300.

    PubMed  CAS  Google Scholar 

  31. Guyatt GH, Kirschner B, Jaeschke R. Measuring health status: What are the necessary measurement properties? J Clin Epidemiol 1992; 45: 1341–1345.

    Article  PubMed  CAS  Google Scholar 

  32. Pfennings LEMA, Ploeg HM van der, Cohen L, Polman CH. A comparison of responsiveness indices in multiple sclerosis patients. Qual Life Res 1999; 8: 481–489.

    Article  PubMed  CAS  Google Scholar 

  33. Brennan RL. Elements of Generalizability Theory. Iowa city: ACT publications, 1983.

    Google Scholar 

  34. Crick JE, Brennan RL. Manual for GENOVA: A Generalized Analysis of Variance System. Iowa city: American College Testing Program, 1983.

    Google Scholar 

  35. Kramer MS, Feinstein AR. Clinical biostatistics. LIV. The biostatistics of concordance. Clin Pharmacol Ther 1981; 29: 111–123.

    Article  PubMed  CAS  Google Scholar 

  36. Bartko JJ. On various intraclass correlation reliability coefficients. Psycholog Bull 1976a; 83: 762–765.

    Article  Google Scholar 

  37. Bartko JJ, Carpenter WT. On the methods and theory of reliability. J Nerv Mental Dis 1976; 163: 307–317.

    Article  CAS  Google Scholar 

  38. Rankin G, Stokes M. Reliability of assessment tools in rehabilitation: An illustration of appropriate statistical analyses. Clinic Rehabil 1998; 12: 187–199.

    Article  CAS  Google Scholar 

  39. Lachs MS. The more things change.... J Clin Epidemiol 1993; 46: 1091–1092.

    Article  PubMed  CAS  Google Scholar 

  40. Hébert R, Spiegelhalter DJ, Brayne C. Setting the minimal metrically detectable change on disability rating scales. Arch Phys Med Rehabil 1997; 78: 1305–1308.

    Article  PubMed  Google Scholar 

  41. Schuling J, Greidanus J, Meyboom-de Jong B. Measuring functional status of stroke patients with the sickness impact profile. Disabil Rehabil 1993; 15: 19–23.

    Article  PubMed  CAS  Google Scholar 

  42. Stratford P, Solomon P, Binkley J, Finch E, Gill C. Sensitivity of sickness impact profile items to measure change over time in a low back pain patient group. Spine 1993; 18: 1723–1727.

    PubMed  CAS  Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Rights and permissions

Reprints and permissions

About this article

Cite this article

Beckerman, H., Roebroeck, M., Lankhorst, G. et al. Smallest real difference, a link between reproducibility and responsiveness. Qual Life Res 10, 571–578 (2001). https://doi.org/10.1023/A:1013138911638

Download citation

  • Issue Date:

  • DOI: https://doi.org/10.1023/A:1013138911638

Navigation