Skip to main content
Log in

Validation of a Scale for Assessing Social Validity in Mindfulness-Based Educational Programs

Mindfulness Aims and scope Submit manuscript

Abstract

Objectives

Social validity (SV) is a concept used in intervention research and is concerned with the overall acceptability, relevance, and utility of an intervention to all intervention stakeholders. SV not only takes into account efficacy in respect of the pre-defined study outcomes, but also participants’ perceptions of the intervention as well as the wider social context in which it will be applied. There are a growing number of mindfulness-based educational programs (MBEPs) being empirically evaluated and implemented in educational settings. However, due to a lack of scientifically validated instruments that can assess SV in MBEPs, a systematic evaluation of SV in such programs has not been undertaken to date. The aim of this study was to investigate the psychometric properties of the Social Validity Scale of Mindfulness-Based Programs for Adolescents (Escala de Validez Social de Programas de Mindfulness para Adolescentes—EVSPM-A), composed of 20 items.

Methods

The sample comprised 512 compulsory secondary education and high school students (mean age = 14.5; SD = 1.57) from three Spanish educational centers that had completed an MBEP known as the TREVA Program.

Results

The final version of the scale showed good psychometric properties and factor analyses yielded five factors: global impact-satisfaction, acceptance and viability, individual perceived effectiveness, perceived classroom climate; training feasibility, and applicability of techniques.

Conclusions

The EVSPM-A appears to be a suitable means of assessing SV in MBEPs delivered to adolescents. Using the EVSPM-A to evaluate SV can help improve the implementation and long-term efficacy of MBEPs.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1

Similar content being viewed by others

References

  • Ager, K., Albrecht, N. J., & Cohen, M. (2015). Mindfulness in schools research project: Exploring Student’s perspectives of mindfulness—what are students’ perspectives of learning mindfulness practices at school. Psychology, 6(7), 896–914. https://doi.org/10.4236/psych.2015.67088.

    Article  Google Scholar 

  • Baltzell, A. (2016). Mindfulness and performance. London: Cambridge University Press.

  • Barret, P. M., Short, A. L., & Wescombe, K. (2001). Examining the social validity of the FRIENDS treatment program for anxious children. Behaviour Change, 18(2), 63–67.

    Article  Google Scholar 

  • Beauchemin, J., Hutchins, T. L., & Patterson, F. (2008). Mindfulness meditation may lessen anxiety, promote social skills, and improve academic performance among adolescents with learning disabilities. Complementary Health Practice Review, 13(1), 34–45. https://doi.org/10.1177/1533210107311624.

    Article  Google Scholar 

  • Bergomi, C., Tschacher, W., & Kupper, Z. (2013). The assessment of mindfulness with self-report measures: Existing scales and open issues. Mindfulness, 4(3), 191–202. https://doi.org/10.1007/s12671-012-0110-9.

    Article  Google Scholar 

  • Bluth, K., & Blanton, P. W. (2015). The influence of self-compassion on emotional well-being among early and older adolescent males and females. The Journal of Positive Psychology, 10(3), 219–230. https://doi.org/10.1080/17439760.2014.936967.

    Article  PubMed  PubMed Central  Google Scholar 

  • Bollen, K. A., & Curran, P. J. (2005). Latent curve models: a structural equation perspective. Hoboken, NJ: Wiley.

    Book  Google Scholar 

  • Carmines, E. G., & McIver, J. P. (1981). Analyzing models with unobserved variables: analysis of covariance structures. In G. W. Bornstedt & E. F. Borgatta (Eds.), Social Measurement: Current issues (pp. 112–133). Beverly Hills: Sage.

  • Carroll, R., & St. Peter, C. C. (2014). Methods for assessing social validity of behavioral intervention plans for children with attention deficit hyperactivity disorder. Acta de Investigación Psicológica, 4(3), 1642–1656. https://doi.org/10.1016/S2007-4719(14)70971-8.

    Article  Google Scholar 

  • Carter, S. L. (2009). The social validity manual: a guide to subjective evaluation of behavior interventions. London: Academic Press.

  • Crane, R. S., & Hecht, F. M. (2018). Intervention integrity in mindfulness-based research. Mindfulness, 9, 1370–1380. https://doi.org/10.1007/s12671-018-0886-3.

    Article  PubMed  PubMed Central  Google Scholar 

  • Crane, R. S., Eames, C., Kuyken, W., Hastings, R. P., Williams, J. M. G., Bartley, T., et al. (2013). Development and validation of the mindfulness-based interventions–teaching assessment criteria (MBI: TAC). Assessment, 20(6), 681–688. https://doi.org/10.1177/1073191113490790.

    Article  PubMed  Google Scholar 

  • Da Fonceca, V. (1998). Manual de Observación Psicomotriz. Zaragoza: Inde.

    Google Scholar 

  • Dariotis, J., Miralbal-Beltran, R., Cluxtin-Keller, F., Gould, L. F., Greenberg, M. T., & Mendelson, M. (2016). A qualitative evaluation of student learning and skills use in a school-based mindfulness and yoga program. Mindfulness, 7(1), 76-89. https://doi.org/10.1007/s12671-015-0463-y.

  • Davidson, R. J., Dunne, J., Eccles, J. S., Engle, A., Greenberg, M., Jennings, P., & Vago, D. (2012). Contemplative practices and mental training: prospects for American education. Child Development Perspectives, 6(2), 146–153. https://doi.org/10.1111/j.1750-8606.2012.00240.x.

    Article  Google Scholar 

  • Eklund, K., O’malley, M., & Meyer, L. (2017). Gauging mindfulness in children and youth: school-based applications. Psychology in the Schools, 54(1), 101–114. https://doi.org/10.1002/pits.21983.

    Article  Google Scholar 

  • Emerson, L. M., Leyland, A., Hudson, K., Rowse, G., Hanley, P., & Hugh-Jones, S. (2017). Teaching mindfulness to teachers: a systematic review and narrative synthesis. Mindfulness, 8(5), 1136-1149. https://doi.org/10.1007/s12671-017-0691-4.

  • Felver, J. C., Celis-de Hoyos, C. E., Tezanos, K., & Singh, N. N. (2016). A systematic review of mindfulness-based interventions for youth in school settings. Mindfulness, 7(1), 34–45. https://doi.org/10.1007/s12671-015-0389-4.

    Article  Google Scholar 

  • Flook, L., Smalley, S. L., Kitil, M. J., Galla, B. M., Kaiser-Greenland, S., & Locke, J. (2010). Effects of mindful awareness practices on executive functions in elementary school children. Journal of Applied School Psychology, 26, 70–95. https://doi.org/10.1080/15377900903379125.

    Article  Google Scholar 

  • Forman, K., & Soloff, G. (2014). Preparing for educational leadership (2nd ed.). New York: Pearson Education.

  • Fornell, C., & Larcker, D. F. (1981). Evaluating structural equation models with unobservable variables and measurement error. Journal of Marketing Research, 27, 39–50.

    Article  Google Scholar 

  • Greenberg, M. T., & Harris, A. R. (2012). Nurturing mindfulness in children and youth: Current state of research. Child Development Perspectives, 6, 161–166. https://doi.org/10.1111/j.1750-8606.2011.00215.x.

    Article  Google Scholar 

  • Hanley, G. P. (2012). Functional assessment of problem behavior: dispelling myths, overcoming implementation obstacles, and developing new lore. Behavior Analysis in Practice, 5(1), 54–72. https://doi.org/10.1007/BF03391818.

    Article  PubMed  PubMed Central  Google Scholar 

  • Himelstein, S. (2016). A mindfulness-based approach to working with high-risk adolescents. London: Routledge.

    Google Scholar 

  • Hokke, H., Lievense, P., & Sterkenburg, P. S. (2014). The effects of mindfulness for people with visual and moderate intellectual disability and their caregivers. Journal of Applied Research in Intellectual Disabilities, 27(4), 311.

    Google Scholar 

  • Hu, L., & Bentler, P. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: conventional criteria versus new alternatives. Structural Equation Modeling, 6, 1–55. https://doi.org/10.1080/10705519909540118.

    Article  Google Scholar 

  • Husley, J., & Meyer, G. (2003). The incremental validity of psychological testing and assessment: conceptual, methodological, and statistical issues. Psychological Assessment, 15, 466–455. https://doi.org/10.1037/1040-3590.15.4.446.

    Article  Google Scholar 

  • Jennings, P. A., Brown, J. L., Frank, J. L., Doyle, S., Oh, Y., Davis, R., Rasheed, D., De Weese, A., DeMauro, A. A., Cham, H., & Greenberg, M. T. (2017). Impacts of the CARE for Teachers Program on teachers’ social and emotional competence and classroom interactions. Journal of Educational Psychology. Advance online publication. https://doi.org/10.1037/edu0000187.

  • Jöreskog, K., & Sörbom, D. (1997). Lisrel 8: user’s reference guide. Lincolnwood: Scientific Software International.

    Google Scholar 

  • Kallapiran, K., Koo, S., Kirubakaran, R., & Hancock, K. (2015). Review: effectiveness of mindfulness in improving mental health symptoms of children and adolescents: a meta-analysis. Child and Adolescent Mental Health, 20(4), 182–194. https://doi.org/10.1111/camh.12113.

    Article  PubMed  Google Scholar 

  • Kern, L., & Manz, P. (2004). A look at current validity issues of school-wide behavior support. Behavioral Disorders, 30, 47–59.

    Article  Google Scholar 

  • Klingbeil, D. A., Renshaw, T. L., Willenbrink, J. B., Copek, R. A., Chan, K. T., Haddock, A., et al. (2017a). Mindfulness-based interventions with youth: a comprehensive meta-analysis of group-design studies. Journal of School Psychology, 63, 77–103. https://doi.org/10.1016/j.jsp.2017.03.006.

    Article  PubMed  Google Scholar 

  • Klingbeil, D. A., Fischer, A. J., Renshaw, T. L., Bloomfield, B. S., Polakoff, B., Willenbrink, J. B., et al. (2017b). Effects of mindfulness-based interventions on disruptive behavior: a meta-analysis of single-case research. Psychology in the Schools, 54(1), 70–87. https://doi.org/10.1002/pits.21982.

    Article  Google Scholar 

  • Kuby, A. K., McLean, N., & Allen, K. (2015). Validation of the child and adolescent mindfulness measure (CAMM) with non-clinical adolescents. Mindfulness, 6(6), 1448–1455. https://doi.org/10.1007/s12671-015-0418-3.

    Article  Google Scholar 

  • Kuyken, W., Weare, K., Ukoumunne, O.C Vicary, R., Motton, N., Burnett, R., Cullen, C., Hennelly, S., & Huppert, F. (2013). Effectiveness of the mindfulness in schools programme: non-randomised controlled feasibility study. The British Journal of Psychiatry, 203(2), 126–131; doi: https://doi.org/10.1192/bjp.bp.113.126649.

  • Leonard, N. R., Jha, A. P., Casarjian, B., Goolsarran, B., Garcia, C., Cleland, C. M., Gwadz, M. V., & Massey, Z. (2013). Mindfulness training improves attentional task performance in incarcerated youth: a group randomized controlled intervention trial. Frontiers in Psychology, 4(792), 1–10. https://doi.org/10.3389/fpsyg.2013.00792.

    Article  Google Scholar 

  • Liehr, P., & Diaz, N. (2010). A pilot study examining the effect of mindfulness on depression and anxiety for minority children. Archives of Psychiatric Nursing, 24(1), 69–71. https://doi.org/10.1016/j.apnu.2009.10.001.

    Article  PubMed  Google Scholar 

  • López-González, L. (2010). Disseny i desenvolupament d’un programa de relaxació vivencial aplicada a l’aula. Doctoral thesis, University of Barcelona - (UB). Barcelona, Spain. Available at: http://tdx.cat/bitstream/handle/10803/2364/LLG_TESI.pdf?sequence=1. Accessed 19 March 2019.

  • López-González, L., & Bisquerra, R. (2013). Validación y análisis de una escala breve para evaluar el clima de clase en Educación Secundaria. ISEP SCIENCE, 5, 62–77.

    Google Scholar 

  • López-González, L., Álvarez-González, M., & Bisquerra, R. (2016a). Mindfulness and action research in secondary schools. Gestation of the TREVA program. Revista Interuniversitaria de Formación del Profesorado, 87(30,3), 75–91.

    Google Scholar 

  • López-González, L., Amutio, A., Herrero-Fernández, D., & Bisquerra, R. (2016b). Validation of a relaxation-mindfulness skills and states scale for adolescents. Revista Interuniversitaria de Formación del Profesorado, 87(30,3), 93–105.

    Google Scholar 

  • López-González, L., Amutio, A., Oriol, X., & Bisquerra, R. (2016c). Habits related to relaxation and mindfulness of high school students: influence on classroom climate and academic performance. Journal of Psychodidactics, 21(1), 121.138. https://doi.org/10.1387/RevPsicodidact.13866.

    Article  Google Scholar 

  • López-González, L., Amutio, A., & Herrero-Fernández, D. (2018a). The relaxation-mindfulness competence of secondary and high school students and its influence on classroom climate and academic performance. European Journal of Education and Psychology, 11(1), 5–17.

    Article  Google Scholar 

  • López-González, L., Amutio, A., Oriol, X., Gázquez, J. J., Pérez-Fuentes, M. C., & Molero, M. M. (2018b). Development and validation of the relaxation-mindfulness scale for adolescents (EREMIND-A). Psicothema, 30(2), 224–231.

    PubMed  Google Scholar 

  • Lorenzo-Seva, U., & Ferrando, P. J. (2013). Factor 9.2: a comprehensive program for fitting exploratory and Semiconfirmatory factor analysis and IRT models. Applied Psychological Measurement, 37(6), 497–498.

    Article  Google Scholar 

  • Luiselli, J. K., Worthen, D., Carbonell, L., & Queen, A. H. (2017). Social validity assessment of mindfulness education and practices among high school students. Journal of Applied School Psychology, 33(2), 124–135. https://doi.org/10.1080/15377903.2016.1264531.

    Article  Google Scholar 

  • Marchant, M., Heath, M., & Miramontes, N. Y. (2013). Merging empiricism and humanism role of social validity in the school-wide positive behavior support model. Journal of Positive Behavior Interventions, 15(4), 221–230. https://doi.org/10.1177/1098300712459356.

    Article  Google Scholar 

  • Mendelson, T., Greenberg, M. T., Dariotis, J. K., Gould, L. F., Rhoades, B. L., & Leaf, P. J. (2010). Feasibility and preliminary outcomes of a school-based mindfulness intervention for urban youth. Journal of Abnormal Child Psychology, 38, 985–994. https://doi.org/10.1007/s10802-010-9418-x.

    Article  PubMed  Google Scholar 

  • Metz, S. M., Frank, J. L., Reibel, D., Cantrell, T., Sanders, R., & Broderick, P. C. (2013). The effectiveness of the learning to BREATHE program on adolescent emotion regulation. Research in Human Development, 10(3), 252–272. https://doi.org/10.1080/15427609.2013.818488.

    Article  Google Scholar 

  • Mîndrilă, D. (2010). Maximum likelihood (ML) and diagonally weighted least squares (DWLS) estimation procedures: a comparison of estimation bias with ordinal and multivariate non-normal data. International Journal of Digital Society, 1(1), 60–66.

    Article  Google Scholar 

  • Pallozzi, R., Wertheim, E., Paxton, S., & Ong, B. (2016). Trait mindfulness measures for use with adolescents: a systematic review. Mindfulness. https://doi.org/10.1007/s12671-016-0567-z.

  • Parker, A. E., & Kupersmidt, J. B. (2016). Two universal mindfulness education programs for elementary and middle-school students: master mind and moment. In K. Schonert-Reichl & R. Roeser (Eds.), Handbook of mindfulness in education. Mindfulness in Behavioral Health. New York, NY: Springer. https://doi.org/10.1007/978-1-4939-3506-2_21.

    Chapter  Google Scholar 

  • Rennekamp, R. A., & Arnold, M. E. (2009). What progress, program evaluation? Reflections on a quarter-century of extension evaluation practice. Journal of Extension [On-line], 47(3), Article 3COM1 Available at: https://www.joe.org/joe/2009june/pdf/JOE_v47_3comm1.pdf. Accessed 19 March 2019.

  • Sainio, M., Herkama, S., Turunen, T., Rönkkö, M., Kontio, M., Poskiparta, E., & Salmivalli, C. (2018). Sustainable antibullying program implementation: school profiles and predictors. Scandinavian Journal of Psychology. https://doi.org/10.1111/sjop.12487.

  • Saltzman, A., & Goldin, P. (2008). Mindfulness-based stress reduction for school-age children. In L. A. Greco & S. C. Hayes (Eds.), Acceptance and mindfulness treatments for children and adolescents: a practitioner's guide (pp. 139–161). Oakland, CA, US: New Harbinger Publications.

    Google Scholar 

  • Sanger, K. L., & Dorjee, D. (2015). Mindfulness training for adolescents: a neurodevelopmental perspective on investigating modifications in attention and emotion regulation using event-related brain potentials. Cognitive, Affective, & Behavioral Neuroscience, 15(3), 696–711. https://doi.org/10.3758/s13415-015-0354-7.

    Article  Google Scholar 

  • Sapthiang, S., Van Gordon, W., & Shonin, E. (2019). Mindfulness in schools: a health promotion approach to improving adolescent mental health. International Journal of Mental Health and Addiction, 17, 112-119. https://doi.org/10.1007/s11469-018-0001-y.

  • Schonert-Reichl, K. A., & Roeser, R. W. (2016). Handbook of mindfulness in education. New York: Springer.

  • Semple, R. J., Droutman, V., & Reid, B. A. (2017). Mindfulness goes to school: things learned (so far) from research and real-world experiences. Psychology in the Schools, 54(1), 29–52. https://doi.org/10.1002/pits.21981.

    Article  PubMed  Google Scholar 

  • Shababi, S. (2014). Use of a mindfulness practice to decrease problem behavior and increase engaged time of three students in an elementary school setting. Vancouver: University of British Columbia.

  • Sheinman, N., Levy, S., & Lahav, A. (2011). Mindfulness for a whole school–mindfulness and mind-body skills with children. Unpublished manuscript.

  • Shonani, A., & Slone, M. (2013). Middle school transition from the strengths perspective: young adolescents’ character strengths, subjective well-being, and school adjustment. Journal of Happiness Studies, 14, 1163–1181. https://doi.org/10.1007/s10902-012-9374-y.

    Article  Google Scholar 

  • Shonin, E., Van Gordon, W., & Griffiths, M. D. (2012). The health benefits of mindfulness-based interventions for children and adolescents. Education and Health, 30, 94–97.

    Google Scholar 

  • Steiger, J. H. (2007). Understanding the limitations of global fit assessment in structural equation modeling. Personality and Individual Differences, 42(5), 893–898. https://doi.org/10.1016/j.paid.2006.09.017.

    Article  Google Scholar 

  • Stone, J. R. A. (2014). Mindfulness in schools: taking present practice into account. DECP Debate Edition. Available at: https://northampton.academia.edu/JacquelineStone/Papers. Accessed 19 March 2019.

  • Tan, L. B. G. (2016). A critical review of adolescent mindfulness-based programmes. Clinical Child Psychology and Psychiatry, 21(2), 175–177. https://doi.org/10.1177/1359104515577486.

    Article  Google Scholar 

  • Van Gordon, W., Shonin, E., & Griffiths, M. D. (2015). Towards a second-generation of mindfulness-based interventions. Australia and New Zealand Journal of Psychiatry, 49, 591–591.

    Article  Google Scholar 

  • Wiesner, M., & Schanding, G. T. (2013). Exploratory structural equation modeling, bifactor models, and standard confirmatory factor analysis models: application to the BASC-2 behavioral and emotional screening system teacher form. Journal of School Psychology, 51, 751–763. https://doi.org/10.1016/j.jsp.2013.09.001.

    Article  PubMed  Google Scholar 

  • Wisner, B. (2017). Mindfulness and meditation for adolescents. London: Springer.

  • Wolf, M. M. (1978). Social validity: the case for subjective measurement or how applied behavior analysis is finding its heart. Journal of Applied Behavior Analysis, 11, 203–214. https://doi.org/10.1901/2Fjaba.1978.11-203.

    Article  PubMed  PubMed Central  Google Scholar 

  • Worthen, D., & Luiselli, J. K. (2017). Social validity assessment and intervention evaluation of mindfulness education and practices with high school students. Mindfulness, 8(4), 903–910. https://doi.org/10.1007/s12671-016-0664-z.

    Article  Google Scholar 

  • Zenner, C., Herrnleben-Kurz, S., & Walach, H. (2014). Mindfulness-based interventions in schools—a systematic review and meta-analysis. Frontiers in Psychology, 5, 603. https://doi.org/10.3389/fpsyg.2014.00603.

    Article  PubMed  PubMed Central  Google Scholar 

Download references

Acknowledgments

We wish to express our deepest appreciation to the individuals who voluntarily participated in the project and university employees for their support with the study.

Author information

Authors and Affiliations

Authors

Contributions

LLG and AA conceived and designed the study and wrote and edited all the different versions of the ms and the final paper. DHF conducted data analyses. TS participated in recruiting the sample, and in the administration of the different instruments. WVG contributed to the writing and final editing of the manuscript.

Corresponding author

Correspondence to Alberto Amutio.

Ethics declarations

Ethical Approval

All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional research committee and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards. The study was approved by the Bioethics Committee of the University of Barcelona (Spain).

Conflict of Interest

The authors declare that they have no conflict of interest.

Informed Consent

Informed consent was obtained from all individual participants included in the study.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

López-González, L., Herrero-Fernández, D., Amutio, A. et al. Validation of a Scale for Assessing Social Validity in Mindfulness-Based Educational Programs. Mindfulness 10, 1870–1882 (2019). https://doi.org/10.1007/s12671-019-01138-2

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12671-019-01138-2

Keywords

Navigation