Archives of Physical Medicine and Rehabilitation
Systematic reviewNovel Effect Size Interpretation Guidelines and an Evaluation of Statistical Power in Rehabilitation Research
Section snippets
Search procedure
We searched the Cochrane Database of Systematic Reviews20 for meta-analyses with rehabilitation as a keyword (k=313 reviews, as of June 2019). We included meta-analyses that (1) summarized the effects of an intervention; (2) reported effect sizes and sample sizes; (3) aligned with rehabilitation (ie, targeted functioning among those with health conditions); and (4) were nonpharmacological and nonsurgical. Three meta-analyses were excluded after screening because they did not pertain to an
Results
The 3381 effect sizes were classified as follows: (1) organ functions, n=1920; (2) skills and habits, n=864; (3) representations, n=327; and (4) multicomponent interventions, n=270. Exercise-based interventions were most common among treatments targeting organ function (n=1154, 60.1%). Cognitive training was the most frequently reported intervention targeting skills and habits (n=283, 32.8%). Didactic education was the most frequently reported treatment targeting representations (n=157, 48.0%),
Discussion
We leveraged 3381 effect sizes from rehabilitation studies to establish novel guidelines for interpreting the magnitude of rehabilitation treatment effects. These guidelines are uniquely relevant to rehabilitation research, and should therefore supplant the use of Cohen’s13,14 original guidelines. Second, this study retrospectively evaluated the statistical power of studies investigating rehabilitation treatment effects. Information from this study will enable rehabilitation investigators to
Conclusion
This study presents novel and empirically based interpretation guidelines for small, medium, and large rehabilitation treatment effects. Cohen’s effect size interpretation guidelines,13,14 which have been widely adopted across disciplines, do not accurately describe rehabilitation treatment effects. Furthermore, observed effect sizes differ across intervention categories, indicating that researchers should use category-specific guidelines. This study also demonstrates that rehabilitation
Suppliers
- a.
data.table package; Dowle et al.
- b.
R software; The R Foundation for Statistical Computing.
- c.
pwr package; Champely et al.
Acknowledgments
We thank Christopher Brydges, PhD for providing consultation regarding analyses. We also thank the reviewers for improving the clarity, methodological rigor, and potential impact of this article.
References (38)
- et al.
Advancing the evidence base of rehabilitation treatments: a developmental approach
Arch Phys Med Rehabil
(2012) - et al.
Toward improved evidence standards and methods for rehabilitation: recommendations and challenges
Arch Phys Med Rehabil
(2012) Why rehabilitation research does not work (as well as we think it should)
Arch Phys Med Rehabil
(1995)- et al.
Design and implementation of clinical trials in rehabilitation research
Arch Phys Med Rehabil
(2012) - et al.
Improving power and sample size calculation in rehabilitation trial reports: a methodological assessment
Arch Phys Med Rehabil
(2016) - et al.
Using meta-analyses for comparative effectiveness research
Nurs Outlook
(2012) - et al.
A theory-driven system for the specification of rehabilitation treatments
Arch Phys Med Rehabil
(2019) - et al.
The Rehabilitation Treatment Specification System: implications for improvements in research design, reporting, replication, and synthesis
Arch Phys Med Rehabil
(2019) - et al.
Methods to explain the clinical significance of health status measures
Mayo Clin Proc
(2002) - et al.
Fostering implementation of health services research findings into practice: a consolidated framework for advancing implementation science
Implement Sci
(2009)
Measures of effect size in the reporting of rehabilitation research
Am J Phys Med Rehabil
Sample size calculation in physical medicine and rehabilitation: a systematic review of reporting, characteristics, and results in randomized controlled trials
Arch Phys Med Rehabil
Statistical conclusion validity of rehabilitation research. A quantitative analysis
Am J Phys Med Rehabil
Clinical significance testing in rehabilitation research: what, why, and how?
Phys Ther Rev
Beyond statistical significance: clinical interpretation of rehabilitation research literature
Int J Sports Phys Ther
The effect size statistic: useful in health outcomes research?
J Health Psychol
Statistical power analysis for the behavioural sciences
A power primer
Psychol Bull
Effect sizes, confidence intervals, and confidence intervals for effect sizes
Psychol Sch
Cited by (53)
Robot-assisted early mobilization for intensive care unit patients: Feasibility and first-time clinical use
2024, International Journal of Nursing StudiesEffects of walking interventions in persons with multiple sclerosis—A systematic review
2024, Multiple Sclerosis and Related DisordersCardiorespiratory fitness (V̇O<inf>2peak</inf>) across the adult lifespan in persons with multiple sclerosis and matched healthy controls
2024, Journal of Science and Medicine in SportResponsiveness of the Reaching Performance Scale for Stroke
2023, Archives of Physical Medicine and RehabilitationRehabilitation Treatment Specification System: Identifying Barriers, Facilitators, and Strategies for Implementation in Research, Education, and Clinical Care
2023, Archives of Physical Medicine and Rehabilitation
Parts of this work were supported by grant funding from the National Institutes of Health (P2CHD065702). The views expressed in this article are those of the authors and do not necessarily reflect the official policy or position of the Department of Defense, Department of Veterans Affairs, or the United States government.
Disclosures: none.