Abstract
The development and evaluation of mindfulness-based interventions for a variety of psychological and medical disorders have grown exponentially over the past 20 years. Yet, calls for increasing the rigor of mindfulness research and recognition of the difficulties of conducting research on the topic of mindfulness have also increased. One of the major difficulties is the measurement of mindfulness, with varying definitions across studies and ambiguity with respect to the meaning of mindfulness. There is also concern about the reproducibility of findings given few attempts at replication. The current secondary analysis addressed the issue of reproducibility and robustness of the construct of self-reported mindfulness across two separate randomized clinical trials of mindfulness-based relapse prevention (MBRP), as an aftercare treatment for substance use disorder. Specifically, we tested the robustness of our previously published findings, which identified a latent construct of mindfulness as a significant mediator of the effect of MBRP on reducing craving following treatment. First, we attempted to replicate the findings in a separate randomized clinical trial of MBRP. Second, we conducted sensitivity analyses to test the assumption of the no-omitted confounder bias in a mediation model. The effect of MBRP on self-reported mindfulness and overall mediation effect failed to replicate in a new sample. The effect of self-reported mindfulness in predicting craving following treatment did replicate and was robust to the no-omitted confounder bias. The results of this work shine a light on the difficulties in the measurement of mindfulness and the importance of examining the robustness of findings.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Alsubaie, M., Abbott, R., Dunn, B., Dickens, C., Keil, T. F., Henley, W., & Kuyken, W. (2017). Mechanisms of action in mindfulness-based cognitive therapy (MBCT) and mindfulness-based stress reduction (MBSR) in people with physical and/or psychological conditions: a systematic review. Clinical Psychology Review, 55, 74–91. https://doi.org/10.1016/j.cpr.2017.04.008.
Anheyer, D., Haller, H., Barth, J., Lauche, R., Dobos, G., & Cramer, H. (2017). Mindfulness-based stress reduction for treating low back pain. Annals of Internal Medicine, 166, 799–807. https://doi.org/10.7326/M16-1997.
Baer, R. A., Smith, G. T., Hopkins, J., Krietemeyer, J., & Toney, L. (2006). Using self-report assessment methods to explore facets of mindfulness. Assessment, 13, 27–45. https://doi.org/10.1177/1073191105283504.
Bowen, S., Chawla, N., Collins, S. E., Witkiewitz, K., Hsu, S., Grow, J., et al. (2009). Mindfulness-based relapse prevention for substance use disorders: a pilot efficacy trial. Substance Abuse, 30, 295–305. https://doi.org/10.1080/08897070903250084.
Bowen, S., Chawla, N., & Marlatt, G. A. (2011a). Mindfulness-based relapse prevention for addictive behaviors: a clinician’s guide. New York, NY: Guilford Press.
Bowen, S., Witkiewitz, K., Chawla, N., & Grow, J. (2011b). Integrating mindfulness meditation and cognitive behavioral traditions for the long-term treatment of addictive behaviors. Journal of Clinical Outcomes Management, 18, 473–479.
Bowen, S., Witkiewitz, K., Clifasefi, S. L., Grow, J., Chawla, N., Hsu, S. H., et al. (2014). Relative efficacy of mindfulness-based relapse prevention, standard relapse prevention, and treatment as usual for substance use disorders. JAMA Psychiatry, 71, 547–556. https://doi.org/10.1001/jamapsychiatry.2013.4546.
Bravo, A. J., Pearson, M. R., Wilson, A. D., & Witkiewitz, K. (2018). When traits match states: examining the associations between self-report trait and state mindfulness following a state mindfulness induction. Mindfulness, 9, 199–211. https://doi.org/10.1007/s12671-017-0763-5.
Brown, T.A. (2015). Confirmatory factor analysis for applied research (2nd ed.). New York: Guilford Press.
Browne, M. W., & Cudeck, R. (1993). Alternative ways of assessing model fit. In K. A. Bollen & J. S. Long (Eds.), Testing structural equation models (pp. 136–162). Beverly Hills, CA: Sage.
Carrière, K., Khoury, B., Günak, M. M., & Knäuper, B. (2017). Mindfulness-based interventions for weight loss: a systematic review and meta-analysis. Obesity Reviews, 19, 164–177. https://doi.org/10.1111/obr.12623.
Chawla, N. (2010). Experiential avoidance and substance use relapse. ProQuest Dissertation Publishing & Theses Global.
Chawla, N., Collin, S., Bowen, S., Hsu, S., Grow, J., Douglass, A., & Marlatt, G. A. (2010). The mindfulness-based relapse prevention adherence and competence scale: development, interrater reliability, and validity. Psychotherapy Research, 20, 388–397. https://doi.org/10.1080/10503300903544257.
Chen, F. F., Sousa, K. H., & West, S. G. (2005). Teacher’s corner: testing measurement invariance of second-order factor models. Structural Equation Modeling: A Multidisciplinary Journal, 12, 471–492. https://doi.org/10.1207/s15328007sem1203_7.
Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale: Erlbaum.
Costa, J., Marôco, J., Pinto-Gouveia, J., & Galhardo, A. (2014). Validation of the psychometric properties of Acceptance and Action Questionnaire-II in clinical and nonclinical groups of Portuguese population. International Journal of Psychology and Psychological Therapy, 14, 353–364.
Cox, M. G., Kisbu-Sakarya, Y., Miočević, M., & MacKinnon, D. P. (2013). Sensitivity plots for confounder bias in the single mediator model. Evaluation Review, 37, 405–431. https://doi.org/10.1177/0193841X14524576.
Davidson, R. J., & Kaszniak, A. W. (2015). Conceptual and methodological issues in research on mindfulness and meditation. American Psychologist, 70, 581–592. https://doi.org/10.1037/a0039512.
Dimidjian, S., & Segal, Z. V. (2015). Prospects for a clinical science of mindfulness-based intervention. American Psychologist, 70, 593–620. https://doi.org/10.1037/a0039589.
Flannery, B. A., Volpicelli, J. R., & Pettinati, H. M. (1999). Psychometric properties of the Penn Alcohol Craving Scale. Alcoholism: Clinical and Experimental Research, 23, 1289–1295.
Fritz, M. S., Kenny, D. A., & MacKinnon, D. P. (2016). The combined effects of measurement error and omitting confounders in the single-mediator model. Multivariate Behavioral Research, 51, 681–697. https://doi.org/10.1080/00273171.2016.1224154.
Garland, E. L. (2016). Restructuring reward processing with mindfulness-oriented recovery enhancement: novel therapeutic mechanisms to remediate hedonic dysregulation in addiction, stress, and pain. Annals of the New York Academy of Sciences, 1373, 25–37. https://doi.org/10.1111/nyas.13034.
Gethin, R. (2011). On some definitions of mindfulness. Contemporary Buddhism, 12, 263–279.
Goodman, S. N. (1992). A comment on replication, p-values and evidence. Statistics in Medicine, 11, 875–879.
Grossman, P. (2011). Defining mindfulness by how poorly I think I pay attention during everyday awareness and other intractable problems for psychology’s (re) invention of mindfulness: comment on Brown et al. (2011). Psychological Assessment, 23, 1034-40-6. https://doi.org/10.1037/a0022713
Grossman, P., & Van Dam, N. T. (2011). Mindfulness, by any other name …: trials and tribulations of sati in western psychology and science. Contemporary Buddhism, 12, 219–239. https://doi.org/10.1080/14639947.2011.564841.
Grossman, P., Niemann, L., Schmidt, S., & Walach, H. (2004). Mindfulness-based stress reduction and health benefits: a meta-analysis. Journal of Psychosomatic Research, 57, 35–43. https://doi.org/10.1016/S0022-3999(03)00573-7.
Gu, J., Strauss, C., Bond, R., & Cavanagh, K. (2015). How do mindfulness-based cognitive therapy and mindfulness-based stress reduction improve mental health and wellbeing? A systematic review and meta-analysis of mediation studies. Clincal Psychology Review, 37, 1–12. https://doi.org/10.1016/j.cpr.2015.01.006.
Gu, J., Strauss, C., Crane, C., Barnhofer, T., Karl, A., Cavanagh, K., & Kuyken, W. (2016). Examining the factor structure of the 39-item and 15-item versions of the Five Facet Mindfulness Questionnaire before and after mindfulness-based cognitive therapy for people with recurrent depression. Psychological Assessment, 28, 791–802. https://doi.org/10.1037/pas0000263.
Hayes, S. C., Strosahl, K., Wilson, K., Bissett, R., Pistorello, J., Toarmino, D., et al. (2004). Measuring experiential avoidance: a preliminary test of a working model. The Psychological Record, 54, 553–578.
Holland, P.W. (1988). Causal inference, path analysis, and recursive structural equations models. Sociological Methodology, 18, 449–484. doi:https://doi.org/10.2307/271055.
Hölzel, B. K., Lazar, S. W., Gard, T., Schuman-Olivier, Z., Vago, D. R., & Ott, U. (2011). How does mindfulness meditation work? Proposing mechanisms of action from a conceptual and neural perspective. Perspective on Psychological Science: A Journal of the Association for Psychological Science, 6, 537–559.
Hopwood, T. L., & Schutte, N. S. (2017). A meta-analytic investigation of the impact of mindfulness-based interventions on post traumatic stress. Clinical Psychology Review, 57, 12–20. https://doi.org/10.1016/j.cpr.2017.08.002.
Hu, L. T., & Bentler, P. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: conventional criteria versus new alternatives. Structural Equation Modeling: A Multidisciplinary Journal, 6, 1–55. https://doi.org/10.1080/10705519909540118.
Hunter-Reel, D., McCrady, B., & Hildebrandt, T. (2009). Emphasizing interpersonal factors: an extension of the Witkiewitz and Marlatt relapse model. Addiction, 104, 1281–1290. https://doi.org/10.1111/j.1360-0443.2009.02611.x.
Imai, K., Keele, L., & Tingley, D. (2010). A general approach to causal mediation analysis. Psychological Methods, 15, 309–334.
Judd, C. M., & Kenny, D. A. (1981). Process analysis: estimating mediation in treatment evaluations. Evaluation Review, 5, 602–619. https://doi.org/10.1177/0193841X8100500502.
Kabat-Zinn, J. (2011). Some reflections on the origins of MBSR, skillful means, and the trouble with maps. Contemporary Buddhism, 12, 281–306. https://doi.org/10.1080/14639947.2011.564844.
Kabat-Zinn, J. (2013). Full catastrophe living: using the wisdom of the body and mind to face stress, pain, and illness. New York: Bantam.
Kabat-Zinn, J., Lipworth, L., & Burney, R. (1985). The clinical use of mindfulness meditation for the self-regulation of chronic pain. Journal of Behavioral Medicine, 8, 163–190.
Kazdin, A. E., & Nock, M. K. (2003). Delineating mechanisms of change in child and adolescent therapy: methodological issues and research recommendations. Journal of Child Psychology and Psychiatry, 44, 1116–1129. https://doi.org/10.1111/1469-7610.00195.
Khoury, B., Lecomte, T., Fortin, G., Masse, M., Therien, P., Bouchard, V., et al. (2013). Mindfulness-based therapy: a comprehensive meta-analysis. Clinical Psychology Review, 33, 763–771. https://doi.org/10.1016/j.cpr.2013.05.005.
Kline, R. B. (2016). Principle and practice of structural equation modeling (4th ed.). New York, NY, US: Guilford Press.
Kuyken, W., Warren, F. C., Taylor, R. S., Whalley, B., Crane, C., Bondolfi, G., et al. (2016). Efficacy of mindfulness-based cognitive therapy in prevention of depressive relapse. JAMA Psychiatry, 73, 565. https://doi.org/10.1001/jamapsychiatry.2016.0076.
Langer, E. F. (1989). Mindfulness. Cambridge: De Capo Press.
Levinson, D. B., Stoll, E. L., Kindy, S. D., Merry, H. L., & Davidson, R. J. (2014). A mind you can count on: validating breath counting as a behavioral measure of mindfulness. Frontiers in Psychology, 5, 1202. https://doi.org/10.3389/fpsyg.2014.01202.
Li, W., Howard, M. O., Garland, E. L., McGovern, P., & Lazar, M. (2017). Mindfulness treatment for substance misuse: a systematic review and meta-analysis. Journal of Substance Abuse Treatment, 75, 62–96. https://doi.org/10.1016/j.jsat.2017.01.008.
Lilienfeld, S. O. (2017). Psychology’s replication crisis and the grant culture: righting the ship. Perspectives on Psychological Science: A journal of the Association for Psychological Science, 12, 660–664. https://doi.org/10.1177/1745691616687745.
Louise, S., Fitzpatrick, M., Strauss, C., Rossell, S. L., & Thomas, N. (2017). Mindfulness- and acceptance-based interventions for psychosis: our current understanding and a meta-analysis. Schizophrenia Research, 192, 57–63. https://doi.org/10.1016/j.schres.2017.05.023.
Lutz, A., Jha, A. P., Dunne, J. D., & Saron, C. D. (2015). Investigating the phenomenological matrix of mindfulness-related practices from a neurocognitive perspective. American Psychologist, 70, 632–658. https://doi.org/10.1037/a0039585.
MacKinnon, D. P. (2008). Introduction to statistical mediation analysis. New York: Taylor & Francis Group.
MacKinnon, D. P., & Pirlott, A. G. (2014). Statistical approaches for enhancing causal interpretation of the M to Y relation in mediation analysis. Personality and Social Psychology Review, 19, 30–43. https://doi.org/10.1177/1088868314542878.
MacKinnon, D. P., Lockwood, C. M., Hoffman, J. M., West, S. G., & Sheets, V. (2002). A comparison of methods to test mediation and other intervening variable effects. Psychological Methods, 7, 83–104.
Manuel, J. A., Somohano, V. C., & Bowen, S. (2017). Mindfulness practice and its relationship to the Five-Facet Mindfulness Questionnaire. Mindfulness, 8, 361–367. https://doi.org/10.1007/s12671-016-0605-x.
Marsh, H. W., Wen, Z., & Hau, K.-T. (2004). Structural equation models of latent interactions: evaluation of alternative estimation strategies and indicator construction. Psychological Methods, 9, 275–300. https://doi.org/10.1037/1082-989X.9.3.275.
McDonald, R. P., & Ho, M.-H. R. (2002). Principles and practice in reporting structural equation analyses. Psychological Methods, 7, 64–82.
Medvedev, O. N., Siegert, R. J., Kersten, P., & Krägeloh, C. U. (2017). Improving the precision of the Five Facet Mindfulness Questionnaire using a Rasch approach. Mindfulness. https://doi.org/10.1007/s12671-016-0676-8.
Millsap, R. E., & Kwok, O.-M. (2004). Evaluating the impact of partial factorial invariance on selection in two populations. Psychological Methods, 9, 93–115. https://doi.org/10.1037/1082-989X.9.1.93.
Muthén, L. K., & Muthén, B. O. (1998-2017). Mplus user's guide (8th ed.). Los Angeles, CA: Muthén & Muthén.
Ong, J. C., & Sholtes, D. (2010). A mindfulness-based approach to the treatment of insomnia. Journal of Clinical Psychology, 66, 1175–1184. https://doi.org/10.1002/jclp.20736.
Paterson, J., Medvedev, O. N., Sumich, A., Tautolo, E., Krägeloh, S. U., Sisk, R., McNamara, R., Berk, M., Narayanan, A., & Siegert, R. J. (2017). Distinguishing transient versus stable aspects of depression in New Zealand Pacific Island children using generalizability theory. Journal of Affective Disorders, 227, 698–704.
Piet, J., Würtzen, H., & Zachariae, R. (2012). The effect of mindfulness-based therapy on symptoms of anxiety and depression in adult cancer patients and survivors: a systematic review and meta-analysis. Journal of Consulting and Clinical Psychology, 80, 1007–1020. https://doi.org/10.1037/a0028329.
Segal, Z. V., Williams, J. M. G., & Teasdale, J. D. (2002). Mindfulness-based cognitive therapy for depression: a new approach to preventing relapse. New York, NY, US: Guilford Press.
Tang, Y.-Y., & Leve, L. D. (2016). A translational neuroscience perspective on mindfulness meditation as a prevention strategy. Translational Behavioral Medicine, 6, 63–72. https://doi.org/10.1007/s13142-015-0360-x.
Tapper, K. (2017). Mindfulness and craving: effects and mechanisms. Clinical Psychology Review, 59, 101–117. https://doi.org/10.1016/j.cpr.2017.11.003.
Taylor, D. J., & Muller, K. E. (1996). Bias in linear model power and sample size calculation due to estimating noncentrality. Communications in Statistics: Theory and Methods, 25, 1595–1610. https://doi.org/10.1080/03610929608831787.
Tofighi, D., & Kelley, K. (2016). Assessing omitted confounder bias in multilevel mediation models. Multivariate Behavioral Research, 51, 86–105. https://doi.org/10.1080/00273171.2015.1105736.
Tofighi, D., & MacKinnon, D. P. (2011). RMediation: an R package for mediation analysis confidence intervals. Behavioral Research Methods, 43, 692–700. https://doi.org/10.3758/s13428-011-0076-x.
Tofighi, D., Hsiao, Y. Y., Kruger, E. S., MacKinnon, D. P., Van Horn, M. L., & Witkiewitz, K. (in press). Sensitivity analysis in latent growth curve mediation models. Structural Equation Modeling: A Multidisciplinary Journal.
Valente, M. J., Pelham, W. E., Smyth, H., & MacKinnon, D. P. (2017). Confounding in statistical mediation analysis: what it is and how to address it. Journal of Counseling Psychology, 64, 659–671. https://doi.org/10.1037/cou0000242.
Valeri, L., & Vanderweele, T. J. (2013). Mediation analysis allowing for exposure-mediator interactions and causal interpretation: theoretical assumptions and implementation with SAS and SPSS macros. Psychological Methods, 18, 137–150. https://doi.org/10.1037/a0031034.
Van Dam, N. T., Earleywine, M., & Danoff-Burg, S. (2009). Differential item function across meditators and non-meditators on the Five Facet Mindfulness Questionnaire. Personality and Individual Differences, 47, 516–521. https://doi.org/10.1016/J.PAID.2009.05.005.
Van Dam, N.T., van Vugt, M.K., Vago, D.R., Schmalzl, L., Saron, C.D., Olendzki, A., … Meyer, D.E. (2018). Mind the hype: a critical evaluation and prescriptive agenda for research on mindfulness and meditation. Perspectives on Psychological Science: A Journal of the Association for Psychological Science, 13, 36–61. https://doi.org/10.1177/1745691617709589.
Vandenberg, R. J., & Lance, C. E. (2000). A review and synthesis of the measurement invariance literature: suggestions, practices, and recommendations for organizational research. Organizational Research Methods, 3, 4–70. https://doi.org/10.1177/109442810031002.
van der Velden, A. M., Kuyken, W., Wattar, U., Crane, C., Pallesen, K. J., Dahlgaard, J., et al. (2015). A systematic review of mechanisms of change in mindfulness-based cognitive therapy in the treatment of recurrent major depressive disorder. Clinical Psychological Review, 37, 26–39. https://doi.org/10.1016/j.cpr.2015.02.001.
VanderWeele, T. J. (2010). Bias formulas for sensitivity analysis for direct and indirect effects. Epidemiology, 21, 540–551. https://doi.org/10.1097/EDE.0b013e3181df191c.
Witkiewitz, K., & Bowen, S. (2010). Depression, craving, and substance use following a randomized trial of mindfulness-based relapse prevention. Journal of Consulting and Clinical Psychology, 78, 362–374. https://doi.org/10.1037/a0019172.
Witkiewitz, K., Bowen, S., Douglas, H., & Hsu, S. H. (2013). Mindfulness-based relapse prevention for substance craving. Addictive Behaviors, 38, 1563–1571. https://doi.org/10.1016/j.addbeh.2012.04.001.
Witkiewitz, K., Bowen, S., Harrop, E. N., Douglas, H., Enkema, M., & Sedgwick, C. (2014a). Mindfulness-based treatment to prevent addictive behavior relapse: theoretical models and hypothesized mechanisms of change. Substance Use & Misuse, 49, 513–524. https://doi.org/10.3109/10826084.2014.891845.
Witkiewitz, K., Warner, K., Sully, B., Barricks, A., Stauffer, C., Thompson, B. L., & Luoma, J. B. (2014b). Randomized trial comparing mindfulness-based relapse prevention with relapse prevention for women offenders at a residential addiction treatment center. Substance Use & Misuse, 49, 536–546. https://doi.org/10.3109/10826084.2013.856922.
Young, K. S., van der Velden, A. M., Craske, M. G., Pallesen, K. J., Fjorback, L., Roepstorff, A., & Parsons, C. E. (2018). The impact of mindfulness-based interventions on brain activity: a systematic review of functional magnetic resonance imaging studies. Neuroscience and Biobehavioral Reviews, 84, 424–433. https://doi.org/10.1016/j.neubiorev.2017.08.003.
Funding
The current study was funded by NIAAA R01 AA025539 (Witkiewitz and Tofighi, MPIs) and NIDA R37DA09757 (MacKinnon, PI).
Author information
Authors and Affiliations
Contributions
YYH conducted the data analyses and wrote the results. DT collaborated with the design and execution of the study and conceptualizing the analyses. ESK, MLVH, and DPM collaborated with the analyses and editing of the final manuscript. KW designed and executed the study and wrote the introduction and discussion sections of the manuscript. All authors contributed to the final editing of the manuscript.
Corresponding author
Ethics declarations
Conflict of Interest
The authors declare that they have no conflict of interest.
Ethical Approval
All procedures performed in our study were approved by the institutional review board at the University of Washington.
Informed Consent
Informed consent was obtained from all participants in the present study.
Electronic Supplementary Material
ESM 1
(DOCX 12 kb)
Rights and permissions
About this article
Cite this article
Hsiao, YY., Tofighi, D., Kruger, E.S. et al. The (Lack of) Replication of Self-Reported Mindfulness as a Mechanism of Change in Mindfulness-Based Relapse Prevention for Substance Use Disorders. Mindfulness 10, 724–736 (2019). https://doi.org/10.1007/s12671-018-1023-z
Published:
Issue Date:
DOI: https://doi.org/10.1007/s12671-018-1023-z