Indexed in MEDLINE, SCI & KoreaMed
pISSN 1011-8934   eISSN 1598-6357
Open Access, Peer-reviewed, Weekly
    J Korean Med Sci. 2023 Oct 23;38(41):e333. English.
    Published online Oct 16, 2023.
    https://doi.org/10.3346/jkms.2023.38.e333
    © 2023 The Korean Academy of Medical Sciences.
    Review

    Causes for Retraction in the Biomedical Literature: A Systematic Review of Studies of Retraction Notices

    Soo Young Hwang,1 Dong Keon Yon,2 Seung Won Lee,3 Min Seo Kim,4 Jong Yeob Kim,1 Lee Smith,5 Ai Koyanagi,6,7 Marco Solmi,8,9,10,11,12 Andre F Carvalho,13 Eunyoung Kim,14,15 Jae Il Shin,16,17,18 and John P A Ioannidis19
      • 1Yonsei University College of Medicine, Seoul, Korea.
      • 2Center for Digital Health, Medical Science Research Institute, Kyung Hee University College of Medicine, Seoul, Korea.
      • 3Sungkyunkwan University School of Medicine, Suwon, Korea.
      • 4Samsung Advanced Institute for Health Sciences & Technology (SAIHST), Sungkyunkwan University, Samsung Medical Center, Seoul, Korea.
      • 5Centre for Health Performance and Wellbeing, Anglia Ruskin University Cambridge, Cambridge, UK.
      • 6Research and Development Unit, Parc Sanitari Sant Joan de Déu, CIBERSAM, Barcelona, Spain.
      • 7ICREA, Barcelona, Spain.
      • 8Department of Psychiatry, University of Ottawa, Ottawa, ON, Canada.
      • 9Department of Mental Health, The Ottawa Hospital, Ottawa, ON, Canada.
      • 10Ottawa Hospital Research Institute (OHRI), Clinical Epidemiology Program, University of Ottawa, Ottawa, ON, Canada.
      • 11School of Epidemiology and Public Health, Faculty of Medicine, University of Ottawa, Ottawa, ON, Canada.
      • 12Department of Child and Adolescent Psychiatry, Charité Universitätsmedizin, Berlin, Germany.
      • 13IMPACT - The Institute for Mental and Physical Health and Clinical Translation, School of Medicine, Barwon Health, Deakin University, Geelong, VIC, Australia.
      • 14Department of Health, Social and Clinical Pharmacy, College of Pharmacy, Chung-Ang University, Seoul, Korea.
      • 15The Graduate School of Pharmaceutical Industry Management, Chung-Ang University, Seoul, Korea.
      • 16Department of Pediatrics, Yonsei University College of Medicine, Seoul, Korea.
      • 17The Center for Medical Education Training and Professional Development, Yonsei Donggok Medical Education Institute, Seoul, Korea.
      • 18Severance Underwood Meta-Research Center, Institute of Convergence Science, Yonsei University, Seoul, Korea .
      • 19Meta-Research Innovation Center at Stanford (METRICS) and Departments of Medicine, Epidemiology and Population Health, Biomedical Data Science, and Statistics, Stanford University, Stanford, CA, USA.
    • Address for Correspondence: Eunyoung Kim, PharmD, PhD, BCPS. Department of Health, Social and Clinical Pharmacy, College of Pharmacy, Chung-Ang University, 84 Heukseok-ro, Dangjak-gu, Seoul 06974, Korea. Email: eykimjcb777@cau.ac.kr
      Address for Correspondence: Jae Il Shin, MD, PhD. Department of Pediatrics, Yonsei University College of Medicine, 50 Yonsei-ro, Seodaemun-gu, Seoul 03722, Korea. Email: shinji@yuhs.ac
    Received May 28, 2023; Accepted August 31, 2023.

    This is an Open Access article distributed under the terms of the Creative Commons Attribution Non-Commercial License (https://creativecommons.org/licenses/by-nc/4.0/) which permits unrestricted non-commercial use, distribution, and reproduction in any medium, provided the original work is properly cited.

    Abstract

    Background

    Many studies have evaluated the prevalence of different reasons for retraction in samples of retraction notices. We aimed to perform a systematic review of such empirical studies of retraction causes.

    Methods

    The PubMed/MEDLINE database and the Embase database were searched in June 2023. Eligible studies were those containing sufficient data on the reasons for retraction across samples of examined retracted notices.

    Results

    A 11,181 potentially eligible items were identified, and 43 studies of retractions were included in this systematic review. Studies limited to retraction notices of a specific subspecialty or country, journal/publication type are emerging since 2015. We noticed that the reasons for retraction are becoming more specific and more diverse. In a meta-analysis of 17 studies focused on different subspecialties, misconduct was responsible for 60% (95% confidence interval [CI], 53–67%) of all retractions while error and publication issues contributed to 17% (95% CI, 12–22%) and 9% (95% CI, 6–13%), respectively. The end year of the retraction period in all included studies and the proportion of misconduct presented a weak positive association (coefficient = 1.3% per year, P = 0.002).

    Conclusion

    Misconduct seems to be the most frequently recorded reason for retraction across empirical analyses of retraction notices, but other reasons are not negligible. Greater specificity of causes and standardization is needed in retraction notices.

    Graphical Abstract

    Keywords
    Retraction; Retraction of Publication; Withdrawal; Misconduct

    INTRODUCTION

    The frequency of retractions in scientific journals has been increasing over time.1, 2 There are debates about whether this increase may represent an increase in (suspected) misconduct, rising awareness3 or even the willingness of authors to remove inaccurate papers.4 Some fields with high publication rates such as coronavirus disease 2019 (COVID-19) have also had an “alarmingly high rate of retractions.”5, 6

    Previous studies define retraction as an amendment for a published article to address seriously flawed or erroneous content or data.7 Retraction may address proven misconduct or fully invalidated results, but its boundaries are contested.4 Other forms of amendments include ‘correction/erratum,’ ‘partial retraction,’ ‘expression of concern,’ ‘withdrawal,’ ‘version/edition,’ ‘editor’s note,’ ‘comment,’ and ‘retired.’4, 8

    The concept and application of retractions have been evolving.9 The Office of Research Integrity (ORI) reviewed misconduct in US Public Health Service funded research since 1992.10 The Committee on Publication Ethics’s retraction guidelines for editors were first published in 2009, containing information on when to consider retraction, the form of the retraction, who should issue the retraction, and when to issue the retraction.7 Retraction Watch, initially starting as a blog in 2010 to draw attention to retracted scientific articles, has grown considerably and launched its own database in 2018.10, 11 In 2016, the National Library of Medicine discontinued the identification of the term ‘partial retraction.’12

    Despite the growing attention to retracted publications, this is the first systematic review to analyze studies reviewing retractions. This study aimed to explore the developing interest in retracted papers and the distribution of the reasons for retraction in published empirical analyses that reviewed samples of retraction notices. We systematically reviewed the reason for retraction and the characteristics of the retracted articles by analyzing previous studies on retraction and quantified the relationship between the temporal trend in retraction reasons and the association with the characteristics of retracted articles.

    METHODS

    Literature search strategy and eligibility criteria

    We followed the guideline of Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) 2020 checklist (Supplementary Table 1) and registered the study protocol in advance on PROSPERO (registration number: CRD42022304160).13, 14 We focused on studies searched through the PubMed/MEDLINE database of retraction notices so as to cover the broad biomedical literature. Two researchers (S.Y.H. and J.Y.K.) independently searched PubMed on January 25, 2022, using keywords such as “retraction” and “misconduct”, “fabrication”, “error”, and “duplication” (complete search strategy provided in Supplementary Table 2), and an updated search of PubMed and EMBASE was performed on June 25, 2023, with the same keywords.

    Eligible studies were those that presented an analysis of retracted papers and presented data on the reasons for retraction. Case studies or case series on a limited number (< 10) of retracted papers were excluded. Studies lacking sufficient data on the reason for retraction were excluded. When there were two or more articles on a similar topic, for example, the same subspecialty, the study with a larger study period, the study containing more relevant information, and the study with a bigger number of retracted papers reviewed were prioritized (in that order) and included through the discussion of two researchers (S.Y.H. and J.Y.K.). We aimed to cover broadly all biomedical sciences, but otherwise, the field of study and article type was not limited. The title and abstract, followed by the full text were extracted for all eligible studies included for screening. Citations from the relevant articles and studies through manual search were additionally inspected.

    Classification of reason for retraction

    We adapted the classification commonly applied by the included studies to efficiently conduct a meta-analysis. Classification of the reason for retractions was divided into the following main categories: ‘Misconduct’, ‘Error’, and ‘Publication Issues’. The specified classifications of Gaudino et al.15 2020 and Rapani et al.16 2020, and the Retraction Watch database11 were used as our reference as we developed our classification of the reasons for retraction.

    Misconduct is subdivided into ‘Plagiarism’, ‘Fabrication/Falsification’, ‘Ethical Issues’ (including Lack of IRB/IACUC approval), ‘Duplication/Overlap’, and ‘Other misconduct’. Error is comprehensive of ‘Honest Error’ and ‘Methodological concerns’, and publication issues are subdivided into ‘Publisher Error or Compromised Peer Review’ and ‘Authorship Issues’.

    Data extraction and classification of included studies

    From the eligible studies, we extracted the following items:

    • 1. Name of the first author; publication year; period of retraction; year of oldest publication; field of study; database searched

    • 2. Number of total published articles during the study period corresponding to the journals/fields of the retracted articles; the number of retracted articles included in the study; the number of journals that published the retracted articles; average time-to-retraction (time from publication to retraction)

    • 3. Countries affiliated with the retracted articles (country affiliation of the institution, first author, or corresponding author, defined by the article); publication type of the retracted articles; median impact factor of the retracted articles; citations of the retracted articles (pre-retraction cites, post-retraction cites, total retraction cites) as presented by the original article

    • 4. Reason for retraction and the number of studies or the proportion for each reason.

    In order to more effectively analyze articles on retraction, we implemented a categorization of the included studies as follows.

    • a. Biomedicine; Studies based on a search of PubMed/MEDLINE, a biomedical database

    • b. Medicine; Limited to medicine only

    • c. Field of study subspecialized from biomedicine

    • d. Country of affiliation or the country of the authors

    • e. Publication type or journal type

    • f. Others (including both biomedical and other disciplines)

    Statistical analysis

    Meta-analysis of proportions was performed on the seventeen studies that were specified to cover a certain subspecialty in order to reduce the redundancy of data. We used both a broader categorization (i.e., misconduct, error, and publication issues) and a more detailed categorization of reasons. The fixed (common) effect model and the random effects model were used to obtain the summary estimates and the summary results were displayed in forest plots. Proportions were transformed using the Freeman-Tukey double arcsine transformation17 to more appropriately handle small sample sizes and extreme proportions. Heterogeneity between the included studies was quantified using the I2 statistic.18

    The summary estimates of all included studies and the selected studies were used to conduct a meta-regression analysis. Exploratory meta-regression analyses were performed using continuous variables such as the end of the retraction period and the median impact factor of retracted publications as moderators. Small study effects (which may reflect selective reporting or other reasons) were examined through visual inspection of the funnel plot and Egger’s test.19 All statistical tests were two-sided. Statistical analyses were performed using software R version 4.1.2 (R Foundation, Vienna, Austria) and its “meta” and “metafor” packages.20, 21, 22

    RESULTS

    Through PubMed search, 1,336 potentially eligible items were identified, of which 69 studies were selected for in-depth text screening and 38 studies were included for final analysis. Through an updated search of PubMed and Embase, we identified 10,130 records, of which after initial screening and removal of redundant publications, three more articles were additionally included. Two more additional articles were identified through citation searching. Finally, forty-three articles3, 15, 16, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62 were eligible (Table 1). After a thorough discussion between the authors, the quantitative data of Shimray53 2022 were presented but weren’t merged as it was considered that there wasn’t enough time for misconduct to have been investigated, due to the short time span of COVID-19-related papers published in 2020. The search and selection processes are shown in Supplementary Fig. 1.

    Table 1
    Baseline characteristics of studies included for review

    Characteristics of literature

    The databases mainly searched were PubMed/MEDLINE Database and the Retraction Watch Database. Eleven articles were from the search of retracted articles mainly on PubMed/MEDLINE, a biomedical database, with different study periods starting in 1966 to the latest in 2016. Gaudino et al.15 concentrated on retracted articles of medicine through the Retraction Watch database with ‘medicine’ in the subject code. Eighteen articles limited the search to a more specific subspecialty such as psychiatry, radiology, genetics, and COVID-19. Seven articles focused on the country or continent of affiliations or the country of authors, and four articles focused on a specific publication type or specific journal type. Lastly, two articles were performed more broadly, extending beyond biomedicine to other scholarly fields. Studies limited to a subspecialty, country, or specific publication/journal included a selection and review process.

    Forty-one out of forty-three studies included for review were published after 2010 and studies focusing on a certain subspecialty or country, journal/publication type was published starting from 2015 (Fig. 1). According to Redman et al.,49 5,041,587 studies were published between 1995–2004, of which 328 (0.0065%) were retracted. The mean time-to-retraction ranged from 10.2 months29 to 50.8 months.47 Twenty-nine studies described the country of the author or country affiliated with (Supplementary Table 4), and eighteen studies provided the article type of the retracted articles (Supplementary Table 5). Fourteen studies stated additional information on the mean or median impact factor of retracted papers that ranged from a mean of 1.0329 to a mean of 10.3649 (Supplementary Table 6). Eighteen studies contained information on either pre-, post-retractions, or the total number of citations (Supplementary Table 7).

    Fig. 1
    Bubble plot of all forty-three studies on retracted publications included for review.

    Literature review of the terminology

    As mentioned above, we used three main categories for reasons of retraction: misconduct, error, and publisher issues. Reasons for retraction that does not fit in any of these categories were defined as Others. The boundaries were not fully consistent across the analyzed studies. The reason for retraction in this paper used was extracted and followed the original article, although discrepancies existed in categorizing the specific reasons into the broader three main categories mentioned above. Studies had an agreement in most definitions, but also some diversity in used terminology and its meanings.

    Not all journals specified the reasons for Misconduct, and the usage of the term misconduct by some studies43, 48 was identified as limited to fabrication or falsification. Other reasons for misconduct were ‘undeclared conflict of interest’, ‘image duplication’, ‘no permission for data’, ‘breach of editorial policy’, ‘property or legal concerns’, ‘citation manipulation’, and other unspecified misconduct. Moylan and Kowalczuk44 and Campos-Varela and Ruano-Raviña27 autonomously classified a certain reason for retraction as misconduct, honest error, or unclear boundaries.

    Error contained methodological concerns, honest error, and unreliable or not reproducible data. Budd et al.26 and Redman et al.49 used the expression ‘inability to reproduce’ for cases that did not match confirmed data fabrication/falsification by the US ORI and distinguished it from misconduct and research error.

    Publisher issues contained both publisher’s error and publishing misconduct. Compromised peer review and authorship issues were the most frequently stated reasons associated with publication. There was variation in classifying duplication which is a misconduct of the authors and double publication, an error of the publisher. Grieneisen and Zhang38 and Nair et al.45 classified duplicate publication as publishing misconduct/error, and Rosenkrantz50 and Panahi and Soleimanpour47 provided a detailed description as in ‘inadvertent duplicate publishing of same article’. Rapani et al.16 and Grieneisen and Zhang38 stated both duplication and double publication.

    No reason was available when the reason was either not reported, not available or accessible, or was under investigation. Other reasons for retraction (which we did not include in any of the previous categories) were other unspecified reasons, ‘erroneous retraction’, ‘non-verifiable references’, ‘withdrawn at the request of the author and/or editor’, ‘withdrawn Cochrane reviews’, ‘withdrawal from considerations’, ‘inappropriate comments’, ‘unintended publication’, ‘temporary removal’, ‘not presented at conference’, and ‘concerns/issues about referencing/attributions’.

    Proportion of reasons for retraction

    The total proportion of misconduct ranged from 21.7%53 to 88.6%.45 More specifically for misconduct, plagiarism was the most dominant reason for twenty-three cohorts, duplication/overlap for eleven cohorts, and fabrication for seven cohorts (four studies were divided into two separate cohorts).

    Error ranged from 0.1%27 to 54.9%26. Publication issues ranged from 0.9%45 to 47.6%.43 Three earlier studies3, 26, 49 did not present publication issues as the main reason for any retraction and another one58 presented just 1.0% of the entire retractions due to publication issues.

    Meta-analysis of 17 studies categorized by subspecialties

    The summary estimate of the proportion of overall misconduct in the random effects model was 60% (95% confidence interval [CI], 53–67%), (Fig. 2A). Plagiarism accounted for 15% (95% CI, 11–19%), fabrication/falsification for 19% (95% CI, 12–26%), duplication/overlap for 19% (95% CI, 14–23%), and ethical issues for 7% (95% CI, 3–12%) (Table 2).

    Fig. 2
    Forest plots of reason for retraction in studies categorized by subspecialties. (A) Forest plot of the proportion of misconduct for retraction in studies categorized by subspecialties. (B) Forest plot of the proportion of error for retraction in studies categorized by subspecialties. (C) Forest plot of the proportion of publication issues for retraction in studies categorized by subspecialties.

    Table 2
    Summary effect size of included studies according to subspecialties

    The proportion of error in the random-effects model was 17% (95% CI, 12–22%) and the proportion of publication issues in the random-effects model was 9% (95% CI, 6–13%) (Fig. 2B and C). The proportion of authorship issues was 7% (95% CI, 4–10%) and publisher error/compromised peer review proportion was 6% (95% CI, 4–8%). No reason available accounted for 15% (95% CI, 9–21%) (Table 2).

    Meta-regression

    We performed meta-regression analyses with the proportion of misconduct as the outcome and continuous variable moderators; the end year of the period of retraction, and the median impact factor of the retracted articles. We performed a meta-regression on the 17 selected studies based on subspecialties and 42 studies included for review to compare the temporal trend in retraction. The proportion of misconduct and the end year of the retraction period in the selected studies based on their subspecialties didn’t have a statistically significant correlation (coefficient = 0.63%; 95% CI, −3.03–4.29%; P = 0.735) and all studies included had a weak positive association (coefficient = +1.3% per year; 95% CI, 0.49–2.09%; P = 0.002) (Fig. 3). The median impact factor of retracted articles failed to show a significant association with the proportion of misconduct (coefficient = +0.85%; 95% CI, −1.01–2.71%; P = 0.370) (Supplementary Fig. 2).

    Fig. 3
    Meta-regression plot of the proportion of misconduct for retraction in studies categorized by subspecialties (left) and all included studies (right).

    DISCUSSION

    Studies that review the reasons for retraction among samples of retracted articles are increasing, with many studies limited to retraction notices of a specific subspecialty or country, journal/ publication type emerging since 2015. Retraction reasons are diverse and the complex nomenclature is not always fully consistent. Overall, misconduct was responsible for an estimated proportion of 60% (95% CI, 53–67%) of all retractions while error and publication issues were responsible for respectively 17% (95% CI, 12–22%) and 9% (95% CI, 6–13%) across the merged 17 studies selected by subspecialties. Meta-regression suggested a weak association with a higher percentage of misconduct among more recent evaluations.

    As the total number of retracted publications increases, it is unclear whether retraction is becoming clearly more common or simply more attention is being devoted to it. For example, alongside many criticisms on the increasing rates of retraction, Fanelli63 demonstrated that the number of retractions per retracting journal has not necessarily increased, and although the queries made to the US ORI increased, the frequency of misconduct has not. Steen et al.64 suggested that the shorter time-to-retraction indicates lower barriers to retraction.

    Franzen et al.65 illustrated that the scientific community has a tendency to treat misconduct as the fault of individuals and that there exists a spiral of mistrust in the competitive biomedical field. Some retractions cluster to a few highly-prolific fraudulent producers. For example, Dr. Yoshitaka Fuji has been responsible for over one-third (37%, 131 of 350) of the retracted anesthesiology articles analyzed by Nair et al.45 If so, evaluations of retractions would need to take into account a strong clustering effect in their analyses. An estimated average of 1.97% of scientists admitted to having fabricated, falsified, or modified data or results at least once.66 Not only misconduct, but the proportion of papers written by single-retraction authors increased from 46.0% in 1972–1992 to 63.1% in 1993–2012 of all retracted articles.64

    Verifying scientific misconduct is a very time-consuming process as the process not only includes the authors and editors but also involves institutions or the ORI and becomes much more complex with the involvement of different countries.67 For recent retraction notifications, there may have been insufficient time to investigate associated misconduct. In this regard, it is important to note that retracted COVID-19-related publications analyzed by Shimray53 presented exceptionally low rates of misconduct.

    Some studies have demonstrated a high mean impact factor of articles retracted due to error.3, 33, 42 Errors are more prone to be revealed when published in high-impact journals. Our findings failed to show a significant correlation. However, our meta-regression was underpowered due to a limited number of included evaluations, plus it was an ecological meta-regression (the median impact factor was used rather than the impact factor of each retraction), thus an association could have been easily missed. King et al.40 found that the median impact factor was higher for administration reasons than content-related reasons.46

    Selection of the retraction notice should be prudent, being negotiated with the editor and authors, but also provide clear and direct information to the readers.7 We observed that the terms and reasons for retraction are being more commonly specified over time, but they also have much more variation compared to two early reviews published in 1998 and 2008.26, 49 Use of terminology such as ‘duplicate publications’ varied, as it was considered as either misconduct or publication error with no clear distinction with ‘double publication’ in some cases. Duplicate is when the author self-plagiarizes one’s already published work without visible reference to the previous publication.44, 68 Duplicate publication is against international copyright laws, ethical conduct, and cost-effectiveness as it can result in redundant data.68 The relative seriousness certainly differs from ‘accidental double publication’ by the publisher.

    The term ‘inability to reproduce’ or ‘author(s) could not replicate the results’ can be applied differently because the nature of misconduct and error differs across basic science disciplines and clinical studies. In many fields of basic science, precise reproducibility of individual experiments is not expected and the consistency of the argument is more valued than the specific details in methodology.65, 69 Dal-Ré and Ayuso31 compared medical genetics to non-medical genetics illustrating that fabrication/falsification was statistically significantly more common in non-medical genetics and medical-genetics articles were more frequently investigated.

    The country of the author or the country affiliated with was a factor that was commonly analyzed. Studies explained that the proportion of low- and middle-income countries (LMICs) was greater than in high-income countries or that Asian authors conducted more research that was retracted.16, 24 Analyzed by the retraction of articles per published literature according to total publications from 2013 to 2016, Iran, Egypt, China, and India ranked in the top five.27 A study on the comparison of Chinese and Indian retracted articles showed that Chinese researchers are awarded for publishing in high-profile journals and retraction profiles are probably shaped by this rewarding practice, while most retracted articles from Indian authors are published in journals with no impact factor.46 Nevertheless, 34% of retracted articles by Indian authors were funded by external funding agencies.36 The high frequency of authorship issues and plagiarism in Iranian retracted publications requires more attention from research ethics authorities.43 Plagiarism was also a leading reason for retraction in African authors where Egyptian authors were associated with over one-third of the retracted articles.51 Stretton et al.70 described how plagiarism is more prevalent in LMICs and Rohwer et al.71 presented that LMIC researchers report that guest authorship is widely accepted and common. Distinctively in South Korea, there was a high proportion of duplicate publications, because there was a great deal of attention in Korean medical societies about the matter with nationwide campaigns by Korean medical editors.59

    The emerging availability of large-scale, comprehensive databases of retractions (e.g., as compiled by Retraction Watch) will hopefully facilitate inclusive analyses of retraction reasons and characteristics. As of July 21st, 2022, the Retraction Watch Database includes 37,982 notices and 102 reasons for retraction, with multiple reasons for a single retraction notice.11 The most frequently occurring reasons are “Notice- Limited or No Information” (27%), “Investigation by Journal/Publisher” (17%), “Date of Retraction/Other Unknown” (13%), and “Breach of Policy by Author” (12%). A search of the database with the reasons for retraction used in this paper yielded the following results; “misconduct” (5%), “plagiarism” (12%), “falsification/fabrication” (5%), “duplication” (16%), “ethical violations” (2%), “error” (15%), “fake peer review” (8% of notices), “duplicate publication through error by journal/publisher” (2%), and “authorship issues” (4%).11 Comparisons of the proportion of different reasons between Retraction Watch, single empirical studies of retraction notices, and our meta-epidemiological studies should be very cautious given the different definitions used and different levels of granularity.

    This study has several limitations. As the search for retracted papers is performed on a database, there can be a redundancy of data between the empirical studies of retractions. To minimize redundancy, the main outcome of our meta-analysis merged data from only seventeen studies specified to a certain subspecialty instead of the entire forty studies included for review. However, we still acknowledge that papers may be included in more than one subspecialty, and the proportion that these overlapping studies comprise is uncertain in the merged results. Also, although we conducted a cautious investigation of all studies on retracted papers, some studies of retraction notices often lacked crucial information (e.g. even basic reasons for retraction) and had to be excluded. We should not generalize results beyond biomedicine. Of the 43 retrieved eligible articles, 41 were exclusively on biomedical fields, while 2 included also other disciplines. We did not examine empirical studies on retraction notices mainly in non-biomedical. For example, in Retraction Watch, 40% of retractions are by the Institute of Electrical and Electronics Engineers.72 Furthermore, the meta-analysis is dependent on the accuracy of the data presented by the synthesized studies. Based on the insignificant results of Egger’s test results and visual inspection of funnel plots (Supplementary Fig. 3) and the fact that there is no merit to reporting certain reasons for retraction, we concluded that a publication bias is less likely but there’s always the potential for publication bias. Lastly, retraction notices may not necessarily reflect in accurate detail the nature and scope of the problems with a retracted paper73 and thus some published reasons for retraction may have dubious veracity. Some temporal changes in the reasons for retraction may simply reflect changes in publisher/journal philosophy and legal advice rather than real changes in the prevalence of the categories of reasons.

    CONCLUSIONS

    Acknowledging these caveats, our overview shows the landscape of empirical evaluation of retraction notices and offers insights into the interpretation of retracted publications. Following the increasing interest in retracted articles, the number of reviews and analyses of retraction notices has also increased, with studies more concentrated on a specific discipline, country, and publication type. The terms in the retraction notices are also developing and being specified. Misconduct was the leading cause of all retracted publications, followed by error and publication issues. Standardization of retraction terminology and more detailed specification of reasons for retraction, including standardization across journals and exploration of studies of specific topics and time frames based on this standardization would be helpful in the future.

    SUPPLEMENTARY MATERIALS

    Supplementary Table 1

    PRISMA 2020 checklist

    Click here to view.(62K, doc)

    Supplementary Table 2

    Full search strategy

    Click here to view.(28K, doc)

    Supplementary Table 3

    Proportion of misconduct

    Click here to view.(96K, doc)

    Supplementary Table 4

    Country of retracted publications

    Click here to view.(48K, doc)

    Supplementary Table 5

    Publication type of retracted publications

    Click here to view.(41K, doc)

    Supplementary Table 6

    Impact factor of retracted papers

    Click here to view.(39K, doc)

    Supplementary Table 7

    Citations of retracted papers

    Click here to view.(41K, doc)

    Supplementary Fig. 1

    Flow chart of all studies included for review.

    Click here to view.(274K, doc)

    Supplementary Fig. 2

    Meta-regression plot for median impact factor versus proportion of misconduct (coefficient, +0.85%; P = 0.370).

    Click here to view.(99K, doc)

    Supplementary Fig. 3

    Funnel plot of studies according to subspecialties.

    Click here to view.(47K, doc)

    Notes

    Disclosure:All authors have completed the ICMJE uniform disclosure form at www.icmje.org/disclosure-of-interest/ and declare Marco Solmi received honoraria/has been a consultant for Angelini, Lundbeck, Otsuka.

    Data Availability Statement:Statistical code and the datasets are available from the corresponding author (shinji@yuhs.ac).

    Author Contributions:

    • Conceptualization: Shin JI.

    • Data curation: Hwang SY, Kim JY.

    • Formal analysis: Hwang SY.

    • Investigation: Kim JY.

    • Methodology: Hwang SY.

    • Project administration: Kim E, Shin JI.

    • Supervision: Kim E, Shin JI, Ioannidis JPA.

    • Writing - original draft: Hwang SY, Ioannidis JPA.

    • Writing - review & editing: Hwang SY, Yon DK, Lee SW, Kim MS, Smith L, Koyanagi A, Solmi M, Carvalho AF, Ioannidis JPA.

    References

      1. Cokol M, Ozbay F, Rodriguez-Esteban R. Retraction rates are on the rise. EMBO Rep 2008;9(1):2.
      1. Vuong QH, La VP, Hồ MT, Vuong TT, Ho MT. Characteristics of retracted articles based on retraction data from online sources through February 2019. Sci Ed 2020;7(1):34–44.
      1. Fang FC, Steen RG, Casadevall A. Misconduct accounts for the majority of retracted scientific publications. Proc Natl Acad Sci U S A 2012;109(42):17028–17033.
      1. Fanelli D, Ioannidis JP, Goodman S. Improving the integrity of published science: an expanded taxonomy of retractions and corrections. Eur J Clin Invest 2018;48(4):e12898
      1. Yeo-Teh NSL, Tang BL. An alarming retraction rate for scientific publications on coronavirus disease 2019 (COVID-19). Account Res 2021;28(1):47–53.
      1. Cha Y, Jung W, Seo M, Rahmati M. The emerging pandemic recent: SARS-CoV-2. Life Cycle 2023;3:e2
      1. Kleinert S. Committee on Publication Ethics (COPE). COPE’s retraction guidelines. Lancet 2009;374(9705):1876–1877.
      1. Barbour V, Bloom T, Lin J, Moylan E.. Amending published articles: time to rethink retractions and corrections? F1000Res 2017;6(1960):1960.
      1. Fanelli D. Set up a ‘self-retraction’ system for honest errors. Nature 2016;531(7595):415.
      1. Reynolds SM. ORI findings of scientific misconduct in clinical trials and publicly funded research, 1992–2002. Clin Trials 2004;1(6):509–516.
      1. The retraction watch database. [Updated 2022]. [Accessed March 26, 2022].
        http://retractiondatabase.org/.
      1. Partial retractions: NLM policy change. [Updated 2016]. [Accessed August 23, 2022].
        https://www.nlm.nih.gov/pubs/techbull/so16/so16_partial_retractions_pol_change.html.
      1. Page MJ, McKenzie JE, Bossuyt PM, Boutron I, Hoffmann TC, Mulrow CD, et al. The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. Syst Rev 2021;10(1):89.
      1. Lee SW, Koo MJ. PRISMA 2020 statement and guidelines for systematic review and meta-analysis articles, and their underlying mathematics: Life Cycle Committee Recommendations. Life Cycle 2022;2:e9
      1. Gaudino M, Robinson NB, Audisio K, Rahouma M, Benedetto U, Kurlansky P, et al. Trends and characteristics of retracted articles in the biomedical literature, 1971 to 2020. JAMA Intern Med 2021;181(8):1118–1121.
      1. Rapani A, Lombardi T, Berton F, Del Lupo V, Di Lenarda R, Stacchi C. Retracted publications and their citation in dental literature: a systematic review. Clin Exp Dent Res 2020;6(4):383–390.
      1. Miller JJ. The inverse of the Freeman–Tukey double arcsine transformation. Am Stat 1978;32(4):138.
      1. Higgins JP, Thompson SG. Quantifying heterogeneity in a meta-analysis. Stat Med 2002;21(11):1539–1558.
      1. Egger M, Davey Smith G, Schneider M, Minder C. Bias in meta-analysis detected by a simple, graphical test. BMJ 1997;315(7109):629–634.
      1. Schwarzer G. Meta: an R package for meta-analysis. R News 2007;7(3):40–45.
      1. Viechtbauer W. Conducting meta-analyses in R with the metafor package. J Stat Softw 2010;36(3):1–48.
      1. R Core Team. R: A Language and Environment for Statistical Computing. Vienna, Austria: R Foundation for Statistical Computing; 2013.
      1. Xu SB, Hu G. A cross-disciplinary and severity-based study of author-related reasons for retraction. Account Res 2022;29(8):512–536.
      1. Balhara YP, Mishra A. A study exploring attributes and nature of the retracted literature on mental disorders. Indian J Med Ethics 2015;12(1):30–37.
      1. Bolboacă SD, Buhai DV, Aluaş M, Bulboacă AE. Post retraction citations among manuscripts reporting a radiology-imaging diagnostic method. PLoS One 2019;14(6):e0217918
      1. Budd JM, Sievert M, Schultz TR. Phenomena of retraction: reasons for retraction and citations to the publications. JAMA 1998;280(3):296–297.
      1. Campos-Varela I, Ruano-Raviña A. Misconduct as the main cause for retraction. A descriptive study of retracted publications and their authors. Gac Sanit 2019;33(4):356–360.
      1. Chambers LM, Michener CM, Falcone T. Plagiarism and data falsification are the most common reasons for retracted publications in obstetrics and gynaecology. BJOG 2019;126(9):1134–1140.
      1. Chauvin A, De Villelongue C, Pateron D, Yordanov Y. A systematic review of retracted publications in emergency medicine. Eur J Emerg Med 2019;26(1):19–23.
      1. Dal-Ré R. Analysis of retracted articles on medicines administered to humans. Br J Clin Pharmacol 2019;85(9):2179–2181.
      1. Dal-Ré R, Ayuso C. Reasons for and time to retraction of genetics articles published between 1970 and 2018. J Med Genet 2019;56(11):734–740.
      1. Damineni RS, Sardiwal KK, Waghle SR, Dakshyani MB. A comprehensive comparative analysis of articles retracted in 2012 and 2013 from the scholarly literature. J Int Soc Prev Community Dent 2015;5(1):19–23.
      1. Decullier E, Huot L, Samson G, Maisonneuve H. Visibility of retractions: a cross-sectional one-year study. BMC Res Notes 2013;6(1):238.
      1. Deculllier E, Maisonneuve H. Correcting the literature: improvement trends seen in contents of retraction notices. BMC Res Notes 2018;11(1):490.
      1. Dutta Majumder P, Raman R, Krishnan T, George R. Analysis of retracted articles in the ophthalmic literature. Eye (Lond) 2021;35(12):3384–3388.
      1. Elango B. Retracted articles in the biomedical literature from Indian authors. Scientometrics 2021;126(5):3965–3981.
      1. Elango B. Characteristics of retracted editorial articles in the biomedical literature. Scientometrics 2022;127(3):1431–1438.
      1. Grieneisen ML, Zhang M. A comprehensive survey of retracted articles from the scholarly literature. PLoS One 2012;7(10):e44118
      1. Hwang K, Wu X. Retracted or withdrawn publications in journals relating to plastic surgery. J Craniofac Surg 2018;29(5):1114–1116.
      1. King EG, Oransky I, Sachs TE, Farber A, Flynn DB, Abritis A, et al. Analysis of retracted articles in the surgical literature. Am J Surg 2018;216(5):851–855.
      1. Li G, Kamel M, Jin Y, Xu MK, Mbuagbaw L, Samaan Z, et al. Exploring the characteristics, global distribution and reasons for retraction of published articles involving human research participants: a literature survey. J Multidiscip Healthc 2018;11:39–47.
      1. Madlock-Brown CR, Eichmann D. The (lack of) impact of retraction on citation networks. Sci Eng Ethics 2015;21(1):127–137.
      1. Mansourzadeh MJ, Ghazimirsaeid J, Motamedi N, Najafi A, Abdullahi Abubakar A, Dehdarirad H. A Survey of Iranian retracted publications indexed in PubMed. Iran J Public Health 2021;50(1):188–194.
      1. Moylan EC, Kowalczuk MK. Why articles are retracted: a retrospective cross-sectional study of retraction notices at BioMed Central. BMJ Open 2016;6(11):e012047
      1. Nair S, Yean C, Yoo J, Leff J, Delphin E, Adams DC. Reasons for article retraction in anesthesiology: a comprehensive analysis. Can J Anaesth 2020;67(1):57–63.
      1. Palla IA, Singson M, Thiyagarajan S. A comparative analysis of retracted papers in health sciences from China and India. Account Res 2020;27(7):401–416.
      1. Panahi S, Soleimanpour S. The landscape of the characteristics, citations, scientific, technological, and altmetrics impacts of retracted papers in hematology. Account Res 2023;30(7):363–378.
      1. Rai R, Sabharwal S. Retracted publications in orthopaedics: prevalence, characteristics, and trends. J Bone Joint Surg Am 2017;99(9):e44
      1. Redman BK, Yarandi HN, Merz JF. Empirical developments in retraction. J Med Ethics 2008;34(11):807–809.
      1. Rosenkrantz AB. Retracted publications within radiology journals. AJR Am J Roentgenol 2016;206(2):231–235.
      1. Rossouw TM, Matsau L, van Zyl C. An analysis of retracted articles with authors or co-authors from the African region: possible implications for training and awareness raising. J Empir Res Hum Res Ethics 2020;15(5):478–493.
      1. Shi Q, Wang Z, Zhou Q, Hou R, Gao X, He S, et al. More consideration is needed for retracted non-Cochrane systematic reviews in medicine: a systematic review. J Clin Epidemiol 2021;139:57–67.
      1. Shimray SR. Research done wrong: a comprehensive investigation of retracted publications in COVID-19. Account Res 2023;30(7):393–406.
      1. Singh HP, Mahendra A, Yadav B, Singh H, Arora N, Arora M. A comprehensive analysis of articles retracted between 2004 and 2013 from biomedical literature - a call for reforms. J Tradit Complement Med 2014;4(3):136–139.
      1. Wang J, Ku JC, Alotaibi NM, Rutka JT. Retraction of neurosurgical publications: a systematic review. World Neurosurg 2017;103:809–814.e1.
      1. Wang T, Xing QR, Wang H, Chen W. Retracted publications in the biomedical literature from open access journals. Sci Eng Ethics 2019;25(3):855–868.
      1. Wasiak J, Hamilton DG, Foroudi F, Faggion CM Jr. Surveying retracted studies and notices within the field of radiation oncology. Int J Radiat Oncol Biol Phys 2018;102(3):660–665.
      1. Woolley KL, Lew RA, Stretton S, Ely JA, Bramich NJ, Keys JR, et al. Lack of involvement of medical writers and the pharmaceutical industry in publications retracted for misconduct: a systematic, controlled, retrospective study. Curr Med Res Opin 2011;27(6):1175–1182.
      1. Huh S, Kim SY, Cho HM. Characteristics of retractions from Korean medical journals in the KoreaMed database: a bibliometric analysis. PLoS One 2016;11(10):e0163588
      1. Kardeş S, Levack W, Özkuk K, Atmaca Aydın E, Seringeç Karabulut S. Retractions in rehabilitation and sport sciences journals: a systematic review. Arch Phys Med Rehabil 2020;101(11):1980–1990.
      1. Stavale R, Ferreira GI, Galvão JAM, Zicker F, Novaes MR, Oliveira CM, et al. Research misconduct in health and life sciences research: a systematic review of retracted literature from Brazilian institutions. PLoS One 2019;14(4):e0214272
      1. Kocyigit BF, Akyol A. Analysis of retracted publications in the biomedical literature from Turkey. J Korean Med Sci 2022;37(18):e142
      1. Fanelli D. Why growing retractions are (mostly) a good sign. PLoS Med 2013;10(12):e1001563
      1. Steen RG, Casadevall A, Fang FC. Why has the number of scientific retractions increased? PLoS One 2013;8(7):e68397
      1. Franzen M, Rödder S, Weingart P. Fraud: causes and culprits as perceived by science and the media. Institutional changes, rather than individual motivations, encourage misconduct. EMBO Rep 2007;8(1):3–7.
      1. Fanelli D. How many scientists fabricate and falsify research? A systematic review and meta-analysis of survey data. PLoS One 2009;4(5):e5738
      1. Bauchner H, Fontanarosa PB, Flanagin A, Thornton J. Scientific misconduct and medical journals. JAMA 2018;320(19):1985–1987.
      1. Recommendations for the conduct, reporting, editing, and publication of scholarly work in medical journals. [Updated 2021]. [Accessed April 3, 2022].
        https://www.icmje.org/recommendations/.
      1. Goodstein D. Conduct and misconduct in science. Ann N Y Acad Sci 1995;775(1):31–38.
      1. Stretton S, Bramich NJ, Keys JR, Monk JA, Ely JA, Haley C, et al. Publication misconduct and plagiarism retractions: a systematic, retrospective study. Curr Med Res Opin 2012;28(10):1575–1583.
      1. Rohwer A, Wager E, Young T. Advancing research integrity: a programme to embed good practice in Africa. Pan Afr Med J 2019;33:298.
      1. Brainard J. In: Science. 362. 2018. Rethinking retractions; pp. 390-393.
      1. Grey A, Avenell A, Bolland M. Timeliness and content of retraction notices for publications by a single research group. Account Res 2022;29(6):347–378.
    Cited by
    Crossref 3
    Google Scholar 
    Scopus 3
    Web of Science 3
    MeSH Terms
    Prevalence
    Publications
    Records
    Respect
    Sensitivity and Specificity
    Systematic Review*
    Metrics
    Share
    Figures
    slide
    slide
    slide

    1 / 3

    Tables
    slide
    slide

    1 / 2

    ORCID IDs
    PERMALINK