Skip to main content
SpringerLink
Log in

A Randomized Control Trial of a Digital Health Tool for Safer Firearm and Medication Storage for Patients with Suicide Risk

  • Published:
Prevention Science Aims and scope Submit manuscript

Abstract

Most patients with suicide risk do not receive recommendations to reduce access to lethal means due to a variety of barriers (e.g., lack of provider time, training). Determine if highly efficient population-based EHR messaging to visit the Lock to Live (L2L) decision aid impacts patient-reported storage behaviors. Randomized trial. Integrated health care system serving Denver, CO. Served by primary care or mental health specialty clinic in the 75–99.5th risk percentile on a suicide attempt or death prediction model. Lock to Live (L2L) is a web-based decision aid that incorporates patients’ values into recommendations for safe storage of lethal means, including firearms and medications. Anonymous survey that determined readiness to change: pre-contemplative (do not believe in safe storage), contemplative (believe in safe storage but not doing it), preparation (planning storage changes) or action (safely storing). There were 21,131 patients randomized over a 6-month period with a 27% survey response rate. Many (44%) had access to a firearm, but most of these (81%) did not use any safe firearm storage behaviors. Intervention patients were more likely to be categorized as preparation or action compared to controls for firearm storage (OR = 1.30 (1.07–1.58)). When examining action alone, there were no group differences. There were no statistically significant differences for any medication storage behaviors. Selection bias in those who responded to survey. Efficiently sending an EHR invitation message to visit L2L encouraged patients with suicide risk to consider safer firearm storage practices, but a stronger intervention is needed to change storage behaviors. Future studies should evaluate whether combining EHR messaging with provider nudges (e.g., brief clinician counseling) changes storage behavior.

ClinicalTrials.gov: NCT05288517.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1

Similar content being viewed by others

Data Availability

De-identified trial data are available by request for scientific research only.

References

  • Ando, S., Kasai, K., Matamura, M., Hasegawa, Y., Hirakawa, H., & Asukai, N. (2013). Psychosocial factors associated with suicidal ideation in clinical patients with depression. Journal of Affective Disorders, 151(2), 561–565.

    Article  PubMed  Google Scholar 

  • Andrés, A. R., & Hempstead, K. (2011). Gun control and suicide: The impact of state firearm regulations in the United States, 1995–2004. Health Policy, 101(1), 95–103.

    Article  Google Scholar 

  • Anestis, M. D., Bandel, S. L., Butterworth, S. E., Bond, A. E., Daruwala, S. E., & Bryan, C. J. (2020). Suicide risk and firearm ownership and storage behavior in a large military sample. Psychiatry Research, 291, 113277.

    Article  PubMed  Google Scholar 

  • Barkin, S. L., Finch, S. A., Ip, E. H., Scheindlin, B., Craig, J. A., Steffes, J., Weiley, V., Slora, E., Altman, D., & Wasserman, R. C. (2008). Is office-based counseling about media use, timeouts, and firearm storage effective? Results from a cluster-randomized, controlled trial. Pediatrics, 122(1), e15-25. https://doi.org/10.1542/peds.2007-2611

    Article  PubMed  Google Scholar 

  • Beidas, R. S., Ahmedani, B. K., Linn, K. A., Marcus, S. C., Johnson, C., Maye, M., Westphal, J., Wright, L., Beck, A. L., Buttenheim, A. M., Daley, M. F., Davis, M., Elias, M. E., Jager-Hyman, S., Hoskins, K., Lieberman, A., McArdle, B., Ritzwoller, D. P., Small, D. S., & Boggs, J. M. (2021). Study protocol for a type III hybrid effectiveness-implementation trial of strategies to implement firearm safety promotion as a universal suicide prevention strategy in pediatric primary care. Implementation Science, 16(1), 89. https://doi.org/10.1186/s13012-021-01154-8

    Article  PubMed  PubMed Central  Google Scholar 

  • Belsher, B. E., Smolenski, D. J., Pruitt, L. D., Bush, N. E., Beech, E. H., Workman, D. E., Morgan, R. L., Evatt, D. P., Tucker, J., & Skopp, N. A. (2019). Prediction models for suicide attempts and deaths: A systematic review and simulation. JAMA Psychiatry, 76(6), 642–651.

    Article  PubMed  Google Scholar 

  • Betz, M. E., Brooks-Russell, A., Brandspigel, S., Novins, D. K., Tung, G. J., & Runyan, C. (2018). Counseling suicidal patients about access to lethal means: Attitudes of emergency nurse leaders. Journal of Emergency Nursing, 44(5), 499–504.

    Article  PubMed  PubMed Central  Google Scholar 

  • Betz, M. E., Kautzman, M., Segal, D. L., Miller, I., Camargo, C. A., Jr., Boudreaux, E. D., & Arias, S. A. (2018). Frequency of lethal means assessment among emergency department patients with a positive suicide risk screen. Psychiatry Research, 260, 30–35.

    Article  PubMed  Google Scholar 

  • Betz, M. E., Knoepke, C. E., Simpson, S., Siry, B. J., Clement, A., Saunders, T., Johnson, R., Azrael, D., Boudreaux, E. D., Omeragic, F., Adams, L. M., Almond, S., Juarez-Colunga, E., & Matlock, D. D. (2020). An interactive web-based lethal means safety decision aid for suicidal adults (Lock to Live): Pilot randomized controlled trial. Journal of Medical Internet Research, 22(1), e16253. https://doi.org/10.2196/16253

    Article  PubMed  PubMed Central  Google Scholar 

  • Betz, M. E., Knoepke, C. E., Siry, B., Clement, A., Azrael, D., Ernestus, S., & Matlock, D. D. (2019). ‘Lock to live’: Development of a firearm storage decision aid to enhance lethal means counselling and prevent suicide. Injury Prevention, 25(Suppl 1), i18–i24. https://doi.org/10.1136/injuryprev-2018-042944

    Article  PubMed  Google Scholar 

  • Boggs, J. M., Quintana, L. M., Beck, A., Clinch, S., Richardson, L., Conley, A., Richards, J. E., & Betz, M. E. (2022). “Lock to Live” for firearm and medication safety: Feasibility and acceptability of a suicide prevention tool in a learning healthcare system. Front Digit Health, 4, 974153. https://doi.org/10.3389/fdgth.2022.974153

    Article  PubMed  PubMed Central  Google Scholar 

  • Boggs, J. M., Quintana, L. M., Powers, J. D., Hochberg, S., & Beck, A. (2020). Frequency of clinicians’ assessments for access to lethal means in persons at risk for suicide. Archives of Suicide Research. https://doi.org/10.1080/13811118.2020.1761917

    Article  PubMed  Google Scholar 

  • Boudreaux, E. D., Larkin, C., Sefair, A. V., Mick, E., Clements, K., Pelletier, L., Yang, C., & Kiefe, C. (2022). Studying the implementation of Zero Suicide in a large health system: Challenges, adaptations, and lessons learned. Contemporary Clinical Trials Communications, 30, 100999. https://doi.org/10.1016/j.conctc.2022.100999

    Article  PubMed  PubMed Central  Google Scholar 

  • Burke, T. A., Ammerman, B. A., & Jacobucci, R. (2019). The use of machine learning in the study of suicidal and non-suicidal self-injurious thoughts and behaviors: A systematic review. Journal of Affective Disorders, 245, 869–884.

    Article  PubMed  Google Scholar 

  • Centers for Disease Control and Prevention, N. C. f. H. S. (2020). Underlying cause of death 1999–2019 on CDC WONDER Online Database. Accessed June 3, 2022. http://wonder.cdc.gov/ucd-icd10.html

  • Clark, K. D., Newell, S., Kenyon, E. A., Karras, E., Simonetti, J. A., Gerrity, M., & Dobscha, S. K. (2021). Firearms storage safety discussions in VA primary care: Staff perspectives. General Hospital Psychiatry, 72, 96–101.

    Article  PubMed  Google Scholar 

  • Coalition, C. F. S. (2022). Colorado firearm safety coalition. Accessed June 3, 2022. https://coloradofirearmsafetycoalition.org/

  • Comtois, K. A., Kerbrat, A. H., DeCou, C. R., Atkins, D. C., Majeres, J. J., Baker, J. C., & Ries, R. K. (2019). Effect of augmenting standard care for military personnel with brief caring text messages for suicide prevention: A randomized clinical trial. JAMA Psychiatry, 76(5), 474–483.

    Article  PubMed  PubMed Central  Google Scholar 

  • Crifasi, C. K., Doucette, M. L., McGinty, E. E., Webster, D. W., & Barry, C. L. (2018). Storage practices of US gun owners in 2016. American Journal of Public Health, 108(4), 532–537.

    Article  PubMed  PubMed Central  Google Scholar 

  • Fonseka, T. M., Bhat, V., & Kennedy, S. H. (2019). The utility of artificial intelligence in suicide risk prediction and the management of suicidal behaviors. Australian & New Zealand Journal of Psychiatry, 53(10), 954–964.

    Article  Google Scholar 

  • Hawton, K. (2002). United Kingdom legislation on pack sizes of analgesics: Background, rationale, and effects on suicide and deliberate self-harm. Suicide and Life-Threatening Behavior, 32(3), 223–229.

    Article  PubMed  Google Scholar 

  • Hawton, K., Simkin, S., Deeks, J., Cooper, J., Johnston, A., Waters, K., Arundel, M., Bernal, W., Gunson, B., & Hudson, M. (2004). UK legislation on analgesic packs: Before and after study of long term effect on poisonings. BMJ, 329(7474), 1076.

    Article  PubMed  PubMed Central  Google Scholar 

  • Katon, W., Von Korff, M., Lin, E., Unützer, J., Simon, G., Walker, E., Ludman, E., & Bush, T. (1997). Population-based care of depression: Effective disease management strategies to decrease prevalence. General Hospital Psychiatry, 19(3), 169–178.

    Article  CAS  PubMed  Google Scholar 

  • Kessler, R. C., Bernecker, S. L., Bossarte, R. M., Luedtke, A. R., McCarthy, J. F., Nock, M. K., Pigeon, W. R., Petukhova, M. V., Sadikova, E., & VanderWeele, T. J. (2019). The role of big data analytics in predicting suicide. Personalized Psychiatry, 77–98.

  • Kessler, R. C., Bossarte, R. M., Luedtke, A., Zaslavsky, A. M., & Zubizarreta, J. R. (2020). Suicide prediction models: A critical review of recent research with recommendations for the way forward. Molecular Psychiatry, 25(1), 168–179.

    Article  PubMed  Google Scholar 

  • Knipe, D. W., Chang, S.-S., Dawson, A., Eddleston, M., Konradsen, F., Metcalfe, C., & Gunnell, D. (2017). Suicide prevention through means restriction: Impact of the 2008–2011 pesticide restrictions on suicide in Sri Lanka. PLoS ONE, 12(3), e0172893.

    Article  PubMed  PubMed Central  Google Scholar 

  • Légaré, F., O’Connor, A. C., Graham, I., Saucier, D., Côté, L., Cauchon, M., & Paré, L. (2006). Supporting patients facing difficult health care decisions: Use of the Ottawa Decision Support Framework. Canadian Family Physician, 52(4), 476–477.

    PubMed  PubMed Central  Google Scholar 

  • Leonard, T. C. (2008). Richard H. Thaler, Cass R. Sunstein, Nudge: Improving decisions about health, wealth, and happiness. In: Springer.

  • Loftin, C., McDowall, D., Wiersema, B., & Cottey, T. J. (1991). Effects of restrictive licensing of handguns on homicide and suicide in the District of Columbia. New England Journal of Medicine, 325(23), 1615–1620.

    Article  CAS  PubMed  Google Scholar 

  • Louzon, S. A., Bossarte, R., McCarthy, J. F., & Katz, I. R. (2016). Does suicidal ideation as measured by the PHQ-9 predict suicide among VA patients? Psychiatric Services, 67(5), 517–522.

    Article  PubMed  Google Scholar 

  • McClure, R. J., Turner, C., Peel, N., Spinks, A., Eakin, E., & Hughes, K. (2005). Population-based interventions for the prevention of fall-related injuries in older people. Cochrane Database of Systematic Reviews (1).

  • Motto, J. A., & Bostrom, A. G. (2001). A randomized controlled trial of postcrisis suicide prevention. Psychiatric Services, 52(6), 828–833.

    Article  CAS  PubMed  Google Scholar 

  • Page, A., Liu, S., Gunnell, D., Astell-Burt, T., Feng, X., Wang, L., & Zhou, M. (2017). Suicide by pesticide poisoning remains a priority for suicide prevention in China: Analysis of national mortality trends 2006–2013. Journal of Affective Disorders, 208, 418–423.

    Article  PubMed  Google Scholar 

  • Prochaska, J. O., & DiClemente, C. C. (1982). Transtheoretical therapy: Toward a more integrative model of change. Psychotherapy: Theory, Research & Practice, 19(3), 276.

    Article  Google Scholar 

  • Prochaska, J. O., & DiClemente, C. C. (1983). Stages and processes of self-change of smoking: Toward an integrative model of change. Journal of Consulting and Clinical Psychology, 51(3), 390.

    Article  CAS  PubMed  Google Scholar 

  • Rowhani-Rahbar, A., Simonetti, J. A., & Rivara, F. P. (2016). Effectiveness of interventions to promote safe firearm storage. Epidemiologic Reviews, 38(1), 111–124. https://doi.org/10.1093/epirev/mxv006

    Article  PubMed  Google Scholar 

  • Runyan, C. W., Brooks-Russell, A., Tung, G., Brandspigel, S., Betz, M. E., Novins, D. K., & Agans, R. (2018). Hospital emergency department lethal means counseling for suicidal patients. American Journal of Preventive Medicine, 54(2), 259–265.

    Article  PubMed  Google Scholar 

  • Sale, E., Hendricks, M., Weil, V., Miller, C., Perkins, S., & McCudden, S. (2018). Counseling on Access to Lethal Means (CALM): An evaluation of a suicide prevention means restriction training program for mental health providers. Community Mental Health Journal, 54(3), 293–301. https://doi.org/10.1007/s10597-017-0190-z

    Article  PubMed  Google Scholar 

  • Senore, C., Giordano, L., Bellisario, C., Di Stefano, F., & Segnan, N. (2012). Population based cancer screening programmes as a teachable moment for primary prevention interventions. A review of the literature. Frontiers in Oncology, 2, 45.

    Article  PubMed  PubMed Central  Google Scholar 

  • Shelef, L., Tatsa-Laur, L., Derazne, E., Mann, J. J., & Fruchter, E. (2016). An effective suicide prevention program in the Israeli Defense Forces: A cohort study. European Psychiatry, 31, 37–43.

    Article  CAS  PubMed  Google Scholar 

  • Simon, G. E., Johnson, E., Lawrence, J. M., Rossom, R. C., Ahmedani, B., Lynch, F. L., Beck, A., Waitzfelder, B., Ziebell, R., Penfold, R. B., & Shortreed, S. M. (2018). Predicting suicide attempts and suicide deaths following outpatient visits using electronic health records. American Journal of Psychiatry, 175(10), 951–960. https://doi.org/10.1176/appi.ajp.2018.17101167

    Article  PubMed  Google Scholar 

  • Simon, T. R., Swann, A. C., Powell, K. E., Potter, L. B., Kresnow, M.-J., & O’Carroll, P. W. (2001). Characteristics of impulsive suicide attempts and attempters. Suicide and Life-Threatening Behavior, 32(Supplemental to Issue 1), 49–59.

    CAS  PubMed  Google Scholar 

  • Siry, B. J., Betz, M. E., Azrael, D., Knoepke, C. E., Simpson, S., Omeragic, F., & Matlock, D. D. (2020). A pilot trial of suicidal adults within emergency departments: Enrollment challenges and strategies related to firearm access. SAGE Publications Ltd.

  • Smith, P. M., Cameron, R., McDonald, P. W., Kawash, B., Madill, C., & Brown, K. S. (2004). Telephone counseling for population-based smoking cessation. American Journal of Health Behavior, 28(3), 231–241.

    Article  PubMed  Google Scholar 

  • (SPRC), S. P. R. C. (2020). Counseling on access to lethal means. Accessed June 3, 2022 https://www.sprc.org/resources-programs/calm-counseling-access-lethal-means

  • Stanley, B., & Brown, G. (2021). Stanley brown safety planning intervention. Accessed June 3, 2022. https://suicidesafetyplan.com/

  • Stanley, B., Brown, G. K., Brenner, L. A., Galfalvy, H. C., Currier, G. W., Knox, K. L., Chaudhury, S. R., Bush, A. L., & Green, K. L. (2018). Comparison of the safety planning intervention with follow-up vs usual care of suicidal patients treated in the emergency department. JAMA Psychiatry, 75(9), 894–900. https://doi.org/10.1001/jamapsychiatry.2018.1776

    Article  PubMed  PubMed Central  Google Scholar 

  • Stene-Larsen, K., & Reneflot, A. (2019). Contact with primary and mental health care prior to suicide: A systematic review of the literature from 2000 to 2017. Scandinavian Journal of Public Health, 47(1), 9–17.

    Article  PubMed  Google Scholar 

  • Vroom, E. B., & Massey, O. T. (2021). Moving from implementation science to implementation practice: The need to solve practical problems to improve behavioral health services. The Journal of Behavioral Health Services & Research. https://doi.org/10.1007/s11414-021-09765-1

    Article  Google Scholar 

  • Walby, F. A., Myhre, M. Ø., & Kildahl, A. T. (2018). Contact with mental health services prior to suicide: A systematic review and meta-analysis. Psychiatric Services, 69(7), 751–759.

    Article  PubMed  Google Scholar 

  • Zalsman, G., Hawton, K., Wasserman, D., van Heeringen, K., Arensman, E., Sarchiapone, M., Carli, V., Hoschl, C., Barzilay, R., Balazs, J., Purebl, G., Kahn, J. P., Saiz, P. A., Lipsicas, C. B., Bobes, J., Cozman, D., Hegerl, U., & Zohar, J. (2016). Suicide prevention strategies revisited: 10-year systematic review. Lancet Psychiatry, 3(7), 646–659. https://doi.org/10.1016/S2215-0366(16)30030-X

    Article  PubMed  Google Scholar 

Download references

Funding

Kaiser Permanente’s Office of Community Health as part of their Firearm Injury Prevention Program. The Kaiser Permanente Office of Community Health had no role in the design or conduct of the study; collection, management, analysis, or interpretation of the data; preparation of the manuscript; or the decision to submit the manuscript for publication. Per contract, the office reviewed the manuscript prior to submission for publication, but did not request (and cannot require) any changes.

Author information

Authors and Affiliations

  1. Kaiser Permanente Colorado, Institute for Health Research, 2550 S Parker Rd., Aurora, CO, 80014, USA

    Jennifer M. Boggs, LeeAnn M. Quintana, Arne Beck & Christina L. Clarke

  2. Department of Behavioral Health Services, Kaiser Permanente Colorado, 10350 E Dakota Ave. #125, Denver, CO, 80247, USA

    Laura Richardson, Amy Conley & Edward T. Buckingham

  3. Colorado Permanente Medical Group, Kaiser Permanente Colorado, 1835 Franklin St., Denver, CO, 80218, USA

    Edward T. Buckingham

  4. Kaiser Permanente Washington Health Research Institute, 1730 Minor Ave., Seattle, WA, 98101, USA

    Julie E. Richards

  5. Department of Emergency Medicine, University of Colorado School of Medicine, 12505 E. 16th Ave., Anschutz Inpatient Pav. 2, 1st floor, Aurora, CO, 80045, USA

    Marian E. Betz

Authors
  1. Jennifer M. Boggs
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. LeeAnn M. Quintana
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Arne Beck
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Christina L. Clarke
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Laura Richardson
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Amy Conley
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Edward T. Buckingham
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Julie E. Richards
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Marian E. Betz
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jennifer M. Boggs.

Ethics declarations

Ethics Approval

This study was reviewed and approved by the Kaiser Permanente Institutional Review Board. All patients who completed surveys provided informed consent. This study was performed in accordance with the ethical standards of the 1964 Declaration of Helsinki and its later amendments.

Consent to Participate

Informed consent was obtained from all individual participants included in the study.

Conflict of Interest

The authors declare no competing interests.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Below is the link to the electronic supplementary material.

Supplementary file1 (DOCX 37 KB)

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Boggs, J.M., Quintana, L.M., Beck, A. et al. A Randomized Control Trial of a Digital Health Tool for Safer Firearm and Medication Storage for Patients with Suicide Risk. Prev Sci 25, 358–368 (2024). https://doi.org/10.1007/s11121-024-01641-6

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11121-024-01641-6

Keywords

Search

Navigation