Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

  • Article
  • Published:

Clinical Research

Credibility of ChatGPT in the assessment of obesity in type 2 diabetes according to the guidelines

International Journal of Obesity volume 48, pages 271–275 (2024)Cite this article

Subjects

Abstract

Background

The Chat Generative Pre-trained Transformer (ChatGPT) allows students, researchers, and patients in the medical field to access information easily and has gained attention nowadays. We aimed to evaluate the credibility of ChatGPT according to the guidelines for the assessment of obesity in type 2 diabetes (T2D), which is one of the major concerns of this century.

Materials and method

In this cross-sectional non-human subject study, experienced endocrinologists posed 20 questions to ChatGPT in subsections, which were assessments and different treatment options for obesity according to the American Diabetes Association and American Association of Clinical Endocrinology guidelines. The responses of ChatGPT were classified into four categories: compatible, compatible but insufficient, partially incompatible and incompatible with the guidelines.

Results

ChatGPT demonstrated a systematic approach to answering questions and recommended consulting a healthcare provider to receive personalized advice based on the specific health needs and circumstances of patients. The compatibility of ChatGPT with the guidelines was 100% in the assessment of obesity in type 2 diabetes; however, it was lower in the therapy sections, which included nutritional, medical, and surgical approaches to weight loss. Furthermore, ChatGPT required additional prompts for responses that were evaluated as “compatible but insufficient” to provide all the information in the guidelines.

Conclusion

The assessment and management of obesity in T2D are highly individualized. Despite ChatGPT’s comprehensive and understandable responses, it should not be used as a substitute for healthcare professionals’ patient-centered approach.

This is a preview of subscription content, access via your institution

Access options

Buy this article

  • Purchase on Springer Link
  • Instant access to full article PDF
Buy now

Prices may be subject to local taxes which are calculated during checkout

Similar content being viewed by others

Data availability

The datasets generated during and/or analyzed during the current study are available from the corresponding author on reasonable request.

References

  1. Zimmet P, Alberti KG, Shaw J. Global and societal implications of the diabetes epidemic. Nature. 2001;414:782–7.

    Article  PubMed  CAS  Google Scholar 

  2. ElSayed NA, Aleppo G, Aroda VR, Bannuru RR, Brown FM, Bruemmer D, et al. 8. Obesity and weight management for the prevention and treatment of type 2 diabetes: standards of care in diabetes—2023. Diabetes Care. 2022;46:S128–39.

    Article  PubMed Central  Google Scholar 

  3. Narayan KM, Boyle JP, Thompson TJ, Gregg EW, Williamson DF. Effect of BMI on lifetime risk for diabetes in the U.S. Diabetes Care. 2007;30:1562–6.

    Article  PubMed  CAS  Google Scholar 

  4. Goldstein DJ. Beneficial health effects of modest weight loss. Int J Obes Relat Metab Disord. 1992;16:397–415.

    PubMed  CAS  Google Scholar 

  5. Pastors JG, Warshaw H, Daly A, Franz M, Kulkarni K. The evidence for the effectiveness of medical nutrition therapy in diabetes management. Diabetes Care. 2002;25:608–13.

    Article  PubMed  Google Scholar 

  6. Sarma S, Sockalingam S, Dash S. Obesity as a multisystem disease: trends in obesity rates and obesity-related complications. Diabetes Obes Metab. 2021;23:3–16.

    Article  PubMed  CAS  Google Scholar 

  7. Kung TH, Cheatham M, Medenilla A, Sillos C, De Leon L, Elepaño C, et al. Performance of ChatGPT on USMLE: potential for AI-assisted medical education using large language models. PLoS Digital Health. 2023;2:e0000198.

    Article  PubMed  PubMed Central  Google Scholar 

  8. Xue VW, Lei P, Cho WC. The potential impact of ChatGPT in clinical and translational medicine. Clin Transl Med. 2023;13:e1216.

    Article  PubMed  PubMed Central  Google Scholar 

  9. Christiano PF, Leike J, Brown T, Martic M, Legg S, Amodei D. Deep reinforcement learning from human preferences. Adv Neural Inf Process Syst. 2017. p. 30.

  10. Commission E, Directorate-General for Communications Networks C, Technology. Ethics guidelines for trustworthy AI. Publications Office; 2019.

  11. Dave T, Athaluri SA, Singh S. ChatGPT in medicine: an overview of its applications, advantages, limitations, future prospects, and ethical considerations. Front Artif Intell. 2023;6:1169595.

    Article  PubMed  PubMed Central  Google Scholar 

  12. Yeo YH, Samaan JS, Ng WH, Ting PS, Trivedi H, Vipani A, et al. Assessing the performance of ChatGPT in answering questions regarding cirrhosis and hepatocellular carcinoma. Clin Mol Hepatol. 2023;29:721–32.

    Article  PubMed  PubMed Central  Google Scholar 

  13. Eysenbach G, Powell J, Kuss O, Sa E-R. Empirical studies assessing the quality of health information for consumers on the world wide web: a systematic review. JAMA. 2002;287:2691–700.

    Article  PubMed  Google Scholar 

  14. Homolak J. Opportunities and risks of ChatGPT in medicine, science, and academic publishing: a modern Promethean dilemma. Croat Med J. 2023;64:1–3.

    Article  PubMed  PubMed Central  Google Scholar 

  15. Arif TB, Munaf U, Ul-Haque I. The future of medical education and research: is ChatGPT a blessing or blight in disguise? Med Educ Online. 2023;28:2181052.

    Article  PubMed  PubMed Central  Google Scholar 

  16. Samson SL, Vellanki P, Blonde L, Christofides EA, Galindo RJ, Hirsch IB, et al. American association of clinical endocrinology consensus statement: comprehensive type 2 diabetes management algorithm—2023 update. Endocr Pract. 2023;29:305–40.

    Article  PubMed  Google Scholar 

  17. Roosan D, Samore M, Jones M, Livnat Y, Clutter J. Big-data based decision-support systems to improve clinicians’ cognition. 2016 IEEE International Conference on Healthcare Informatics (ICHI). IEEE; 2016. p. 285–8.

  18. Howard A, Hope W, Gerada A. ChatGPT and antimicrobial advice: the end of the consulting infection doctor? Lancet Infect Dis. 2023;23:405–6.

    Article  PubMed  Google Scholar 

  19. Haver HL, Ambinder EB, Bahl M, Oluyemi ET, Jeudy J, Yi PH. Appropriateness of breast cancer prevention and screening recommendations provided by ChatGPT. Radiology. 2023;307:e230424.

    Article  PubMed  Google Scholar 

  20. Sarraju A, Bruemmer D, Van Iterson E, Cho L, Rodriguez F, Laffin L. Appropriateness of cardiovascular disease prevention recommendations obtained from a popular online chat-based artificial intelligence model. JAMA. 2023;329:842–4.

    Article  PubMed  PubMed Central  Google Scholar 

  21. Schulte B. Capacity of ChatGPT to identify guideline-based treatments for advanced solid tumors. Cureus. 2023;15:e37938.

    PubMed  PubMed Central  Google Scholar 

  22. Samaan JS, Yeo YH, Rajeev N, Hawley L, Abel S, Ng WH, et al. Assessing the accuracy of responses by the language model ChatGPT to questions regarding bariatric surgery. Obes Surg. 2023;33:1790–6.

    Article  PubMed  PubMed Central  Google Scholar 

  23. Cascella M, Montomoli J, Bellini V, Bignami E. Evaluating the feasibility of ChatGPT in healthcare: an analysis of multiple clinical and research scenarios. J Med Syst. 2023;47:33.

    Article  PubMed  PubMed Central  Google Scholar 

  24. Sng GGR, Tung JYM, Lim DYZ, Bee YM. Potential and pitfalls of ChatGPT and natural-language artificial intelligence models for diabetes education. Diabetes Care. 2023;46:e103–5.

    Article  PubMed  Google Scholar 

  25. Vaishya R, Misra A, Vaish A. ChatGPT: is this version good for healthcare and research? Diabetes Metab Syndr Clin Res Rev. 2023;17:102744.

    Article  CAS  Google Scholar 

  26. Temsah O, Khan SA, Chaiah Y, Senjab A, Alhasan K, Jamal A, et al. Overview of early ChatGPT’s presence in medical literature: insights from a hybrid literature review by ChatGPT and human experts. Cureus. 2023;15:e37281.

  27. Hsu Y-C, Ching Y-H. Generative artificial intelligence in education, part one: the dynamic frontier. TechTrends. 2023;67:603–607.

  28. Shao CY, Li H, Liu XL, Li C, Yang LQ, Zhang YJ, et al. Appropriateness and comprehensiveness of using ChatGPT for perioperative patient education in thoracic surgery in different language contexts: survey study. Interact J Med Res. 2023;12:e46900.

    Article  PubMed  PubMed Central  Google Scholar 

Download references

Acknowledgements

We used the generative AI tool ChatGPT-3.5 to obtain responses to our questions that were prepared for the evaluation of credibility. In addition to this, AI tools were not used in the generation of the manuscript in our study.

Author information

Authors and Affiliations

  1. Department of Endocrinology and Metabolism, Gazi University Faculty of Medicine, Ankara, Turkey

    Tugba Barlas, Alev Eroglu Altinova, Mujde Akturk & Fusun Balos Toruner

Authors
  1. Tugba Barlas
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Alev Eroglu Altinova
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Mujde Akturk
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Fusun Balos Toruner
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

All authors contributed to the study conception and design. The first draft of the manuscript was written by TB, FBT and all authors commented on previous versions of the manuscript. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Tugba Barlas.

Ethics declarations

Competing interests

The authors declare no competing interests.

Additional information

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Barlas, T., Altinova, A.E., Akturk, M. et al. Credibility of ChatGPT in the assessment of obesity in type 2 diabetes according to the guidelines. Int J Obes 48, 271–275 (2024). https://doi.org/10.1038/s41366-023-01410-5

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1038/s41366-023-01410-5

This article is cited by

Search

Advanced search

Quick links