Establishment of an early warning list of SCI-indexed international journals with dual benefits for both scientists and publishers

At the end of 2020, the National Science Library of the Chinese Academy of Sciences (CAS) released the Early Warning List (EWL) of International Journals as guidance for scholars to choose appropriate journals in which to publish. The aims of the study were to analyze the characteristics of medical-related journals in the list and to survey scholars in the field of burn specialty regarding their understanding of and attitude toward the journals in EWL to determine their attitude toward the list and the potential use of this list as a reference for international journal quality control. All 26 early warning medical journals in the EWL are characterized by a high proportion of articles from China, as well as a high rate of retraction. The average 5-year proportion of articles authored by Chinese scholars of 19 journals was over 50%, and three were over 90%. Eight journals each retracted more than 20 articles from 2016 to 2020. The questionnaire survey showed that most burn specialty scholars believed that the establishment of such a list was beneficial for enhancing research integrity and improving journal quality management, as well as selecting journals for publication. Based on our study, we recommend establishing a global list of early warning journals to allow the EWL to be updated regularly and released for free. Peer review could be introduced to boost credibility. The EWL released by CAS can provide important practical data for the 'global alliance of early warning journals' in the future. Citation:


Introduction
The number of sci-tech periodicals has increased dramatically around the world over the last quarter of a century [1] , including many journals of questionable quality and predatory journals [2−11] .This issue raises concerns for the academic community, policy-making and evaluation agencies.To avoid submitting to predatory journals, scientists usually choose journals that are included in widely accepted databases with journal impact factors, but many of these are considered to have highly inflated impact factors [12] .In the past decade, several large-scale retractions concerning research integrity and scholarly misconduct have occurred in the Chinese academic community, which may be influenced by the 'SCI worship' evaluation policy.For example, papers published only in SCI journals may be accepted for professional promotion or for personal incentives [13] .The biomedicine discipline in particular, is a strongly affected sector.The early warning policy is an important part of publishing ethics.Relevant work conducted abroad is represented by the DOAJ Whitelist, Beall's List, and Cabells' Journal Whitelist and Blacklist.DOAJ is an internationally renowned academic journal database and is currently the largest database in the world that includes only open access journals.Although there is a strict access mechanism, it is difficult to completely eliminate predatory journals due to the large volume of the database.Beall's List is a blog that documents cases of what Jeffrey Beall, a now-retired librarian at the University of Colorado Denver, Auraria Library, perceived as 'predatory' OA journals or publishers.Beall's influence was profound and international; however, this kind of personal behavior spurred legal threats and led to the eventual demise of Beall's blacklist in 2017.The Cabells' Journal Blacklist, launched in June 2017, contains over 10,000 predatory journals and continues to grow.It is the only database available for scholars to identify predatory journals, but it is currently not available for free and has low popularity.
In light of these circumstances, the Chinese government has presented multiple ordinances [14−16] to eliminate the 'intrinsic and extrinsic sources' of scholarly misconduct and has proposed establishing an early warning mechanism for academic journals.At the end of 2020, the National Science Library of the Chinese Academy of Sciences (CAS) released the Early Warning List of International Journals (Trial) [17] to guide scholars in selecting journals cautiously.Altogether, 65 journals involving over 10 disciplines were categorized into three risk levels, low, ARTICLE medium, and high risk.Among these, 26 medical-related journals were included.
In the current study, we analyzed 26 early warning SCIindexed medical journals in the CAS early warning list (EWL).We also sent a questionnaire to medical staff in burn fields across China to assess scholars' understanding and attitudes toward the policy of early warning journals, and the impact of the policy to provide a reference for international journal quality management.

Indicator collection method
Data were collected and verified from the Web of Science, Journal Citation Reports (JCR), and journal homepages.
After analyzing the main characteristics of early warning medical journals, the authors distributed questionnaires to experts, readers, and authors in the field of burn specialty in China to explore the applicability of early warning policies and whether there is common understanding of the main characteristics of early warning journals.

Questionnaire for science researchers regarding the policy of 'early warning journals' Sources of investigation respondents
Members of the Chinese Burn Association, experts of the editorial board of the Chinese Journal of Burns (an authoritative journal on burn specialty), readers, and authors in approximately 30 provinces, autonomous regions, and direct municipalities across China were included.

Questionnaire items
The items of the questionnaire were in three categories: (1) basic profiles of respondents, including age, sex, ethnicity, highest diploma, occupation, professional title, type of employer, and working years; (2) factors considered for article publication by the respondents; and (3) respondents' knowledge of the policy of 'early warning journals'.

Methodology of investigation
There were a total of 21 questions in the questionnaire in the form of single-choice questions, multiple-choice questions, and single-line blank-filling questions.The investigation period was June 4−14, 2021.The questionnaire was a Medcon network questionnaire, where the participants clicked a web link through a mobile terminal or PC terminal and filled out the questionnaire anonymously.Logic setting was adopted to ensure the integrity of the provided information.In addition, special personnel were assigned for review.If a stakeholder had any doubt, he or she could inquire to confirm the responses by means of phone calls or e-mails.

Statistical analysis
SPSS23.0 was used to analyze data, and the Kolmogorov-Smirnov test was used to conduct the measurement data normal distribution test.In the test, nonnormally distributed data were expressed by the median quartile method and subjected to the nonparametric Kruskal-Wallis test and the Mann-Whitney U test.Enumeration data were expressed as frequency and composition ratio (%), and the chisquare test or Fisher's exact probability test was used for comparison between groups.The test level was α = 0.05.

Profiles of the medical journals in EWL
Among the 26 medical journals in EWL, four were at the 'high-risk level', 18 were at the 'medium-risk level', and four were at the 'low-risk level'.Five were from the publisher SPAN-DIDOS PUBL LTD ('medium-risk'), four were from the publisher E-CENTURY PUBLISHING CORP (two were 'high risk' and two were 'medium risk'), two were from the publisher SAGE PUBLI-CATIONS ('medium risk'), and two were from the publisher MDPI ('low risk').Each of the remaining 13 publishers had one early warning SCI-indexed journal.Of the 26 journals, 23 were also included in PubMed, including three 'high-risk' journals, 17 'medium-risk' journals, and three 'low-risk' journals.In terms of the JCR quartile in 2020, four were in discipline Q1, 10 were in Q2, and six were in Q3 and Q4 according to their highest quartile.In terms of the CAS quartile in 2020, no journal was in discipline Q1, three were in Q2, nine were in Q3, and 14 were in Q4 according to the highest quartile.In terms of OA journals, 17 were OA journals, one was a non-OA journal, and eight were optional.All journals collected an article processing charge (the charges are published on their websites).In terms of publication cycle, two journals were semimonthly (24 editions a year), 17 were monthly (12 editions a year), one released 52 editions a year, one released nine editions a year, two were bimonthly (six editions a year), one released two editions a year, and two were annual.See Supplemental Table S1.

Representative indicators Impact factor
Our statistics show that the 2019 IF of the 26 journals in EWL was 0.166−5.117[2.58 (1.727−3.422)],among which 12 had an IF higher than 3, accounting for 46.1%.Except for Journal of Clinical Medicine, which was indexed by SCI for only 4 years, the 5-year (2014-2018) IF of the other 25 journals was 0.253-5.397[2.42 (1.91−3.31)],among which 11 had an IF higher than 3, accounting for 44.0%.See Supplemental Table S2.

Self-citation rates
Except for Journal of Clinical Medicine, which was indexed by SCI for only 4 years, the 5-year (2014-2018) self-citation rates of the 25 medical journals in EWL were 0.8%-22.4%,among which 22 had a self-citation rate lower than 10%, accounting for 88%; two had a self-citation rate between 10% and 20%, accounting for 8%; and only one was higher than 20%, accounting for 4%.The 2019 self-citation rates of the 26 medical journals in EWL were 1.1%-31.4%,among which only two had a rate higher than 20%, accounting for 7.7%; three had a rate between 10% and 20%, accounting for 11.5%; 17 had a rate lower than 5%, accounting for 65.4%; and four had a rate between 5% and 10%, accounting for 15.4%.See Supplemental Table S2.

Global total number of articles published and proportion of articles authored by Chinese scholars
Between 2016 and 2020, nine journals each published more than 1000 articles, which was the average value of the annual global total number of articles published by the 26 medical journals in EWL, accounting for 34.6%.Two journals exceeded the average value of 2,000 articles as the annual total number of articles published (with mean values of 4,109.6 and 4,310 articles, respectively).Between 2016 and 2020, the proportion of articles authored by Chinese scholars remained at a high level (18 journals) and witnessed a gradual increasing trend (six journals) among the 26 medical journals in EWL, except for the International Journal of Environmental Research and Public Health and the Journal of Clinical Medicine at lower levels.Nineteen out of 26 medical journals in EWL were characterized by an average 5-year proportion of articles authored by Chinese scholars over 50%, among which three journals had an average 5-year proportion higher than 90%.See Supplemental Table S2.S2.

Retraction of published papers
Follow-up analysis was conducted with the high proportions of articles authored by Chinese scholars, and the high prevalence of retractions was characteristic of the medical journals in EWL.

Characteristics of early warning SCI indexed medical journals at different risk levels
There was no significant difference in the JCR quartile, CAS quartile, OA or not, 2019 IF, 5-year (2014-2018) IF, 2019 selfcitation rate, 5-year (2014-2018) self-citation rate, or 2016-2020 combined retraction rates of the journals among the high-risk, medium-risk, and low-risk levels (p > 0.05).There was a significant difference in the publication cycle, the 2016-2020 global total number of articles published, and the proportion of articles authored by Chinese scholars (p < 0.05).See Table 1.The high proportion of articles authored by Chinese scholars was a major characteristic.

Questionnaire recovery
In total, 424 valid questionnaires were recovered.

Basic profiles of the respondents
The majority of respondents were male, aged 42.0 years on average (22.0-64.0),with the age group of 40 years or younger accounting for the highest proportion.The majority were of Han ethnicity.The highest education levels were a master's degree and a doctorate degree, the majority had a senior professional title, and the employers were mainly colleges and universities, research institutes, and teaching hospitals.The mean value of the respondents' working years was 17.00 years (1.0-40.0years), with the majority having either no more than 10 years or over 20 years of experience.The most common occupation type was clinical doctor.See Supplemental Table S3.

Factors considered by the 424 respondents for article publication
The main factors considered for article publication were, according to ranking, the academic influence of the journal (impact factor), the type of journal (such as statistical source journal, core journal, or SCI indexed journal), journal reputation, and the relevant guiding policies of the state and the employer (such as reward and evaluation requirements).See Fig. 1.

Respondents' awareness of the policy of early warning journals
In most cases, the respondents' awareness level was 'know and understand a little bit'.For the list of early warning journals, it was 'know the EWL but don't know the specific journals' in most cases; for the impact of the list on journal selection, it was 'have impact to some extent' in most cases; for the effect of EWL on boosting research integrity, it was 'have some effect' in most cases; for the effect of EWL on improving journal quality management, it was 'have some effect' in most cases; for the impact of EWL on the desire to publish research outcomes, it was 'have some impact' in most cases; for the impact of EWL on choosing Chinese journals as the first choice, it was 'yes, will consider' in most cases.See Table 2.The main factors for early warning of journals were, according to ranking, low academic quality of journals, a high self-citation rate, the relationship with retractions, and a high proportion of articles authored by Chinese scholars.See Fig. 2.

Discussion
Scholars should be cautious in choosing journals with a high proportion of Chinese papers and retraction Among the 26 early warning SCI indexed medical journals released by CAS at the end of 2020, the characteristics are high proportions of articles authored by Chinese scholars and widespread retractions.Table 1 shows that journals at the 'high-risk level' had the largest total number of articles published and the highest proportion of articles authored by Chinese scholars, which is in line with the risk level.The total number of articles published in journals at the 'low-risk level' was higher than that of journals at the 'medium-risk level', whereas the proportion of articles authored by Chinese scholars was lower than that of 'medium-risk' journals, in line with the risk level.Therefore, a high proportion of articles authored by Chinese scholars was the main characteristic of warning journals.Retraction is essentially self-purification aimed at defending academic truth and integrity [18] .However, it is treated cautiously by publishers because it causes ethical problems related to publication [19] .With regard to cases with a large number of retractions, the journal European Review for Medical and Pharmacological Sciences retracted nearly 200 articles in 5 years and 96 articles in 2020 alone, and MEDICINE, Oncology Letters, OncoTargets and Therapy, and LIFE SCIENCES had retractions that reached a peak in 2020.There is no doubt that problems of research integrity and publication ethics may have emerged.These constitute the reasons that some journals are put on the EWL.  2 show that the main factors that the respondents considered for article publication included journal reputation; more than four-fifths (87.3%) of the respondents stated that EWL had an impact on their article submission choices, and almost all respondents believed that early warning of journals had an effect on boosting research integrity (97.7%) and improving journal quality management (99.1%).All of these findings suggest that scholars accept the mechanism of early warning journals, and this mechanism has good prospects for promotion.

Applicability of the early warning journal policy
As Fig. 2 shows, according to the respondents, the main reasons for some journals being put on the EWL included a high proportion of articles authored by Chinese scholars, retractions, and high self-citation rates.The high proportions of articles authored by Chinese scholars and retractions were in line with the characteristic indicators of medical journals in EWL stated in 4.1, which means there is common ground between the perceptions of the respondents and the early warning analysis of the evaluation institutions.High self-citation rates were chosen as one of the main reasons why journals were included in the EML in Fig. 2, which indicates that scholars are aware of normative academic communication.
As Table 3 shows, 1) two high-risk journals on the 2020 warning list, namely, the International Journal of Clinical and Experimental Pathology and the International Journal of Clinical and Experimental Medicine, with 90% Chinese authors in 2019 and 2020, were delisted from the SCI database in 2021; 2) two medium-risk journals on the 2020 warning list, namely, the American Journal of Translational Research and the Journal of Biomaterials and Tissue Engineering, with 90% Chinese authors during 2019-2022, were still on the warning list issued in 2023; 3) the average Chinese author rate is relatively high for the 2020 low-risk journal Acta Medica Mediterranea, with an annual average of over 80%, and shows an upward trend during 2019-2022; it is still listed in the EWL issued in 2023, and the early warning level has been upgraded to medium-risk; and 4) the rate of Chinese authors of other journals on the 2020 warning list showed a general downward trend during 2019-2022.
In addition, six journals appear repeatedly on the 2021 and 2020 lists, three are repeated on the 2020 and 2023 lists, and most repeated journals have a high rate of Chinese authors.Furthermore, "'he high rate of Chinese authors' is the main feature of early warning medical journals.After being warned, the journal is either delisted from the SCI index or continues to be monitored, or it improves on the basis of the 'high Chinese author rate', indicating that the EWL is applicable, recognized by the academic community, and has achieved significant results.

Limitations
Limited by study time and resources, this study could not investigate all burn specialty professionals across China [20] .Nevertheless, we collected samples from approximately 30 provinces, autonomous regions, and direct municipalities of China.The participants were experts, ordinary doctors, and nurses of the burn specialty, and their attitude was representative of the attitude of the entire burn specialty field toward the policy of early warning journals.We hope that in the future, scitech workers and science editors from more disciplines will join this study series so that we can provide comprehensive practical statistics on the Chinese policy of early warning journals worldwide.

Future perspectives
Many similar blacklists or whitelists have been identified, namely, Beall's List, Cabells' Journal Blacklist, and DOAJ inclusion, but each has its limitations.The annual EWL released by CAS has aroused wide public concern since 2020.The influence of these lists clearly illustrates the importance of the role of blacklists in the academic universe.Based on the mentioned works, we propose to establish a global warning alliance to fulfill the pressing need for information on predatory and questionable quality publishing, integrate and unify the warning monitoring indicators, and regularly update and release the EWL with peer experts' comments for free to provide a reference when choosing journals and quality control standards for journal management.

Conclusions
The EWL released by the CAS has attracted widespread attention in the academic world.From our analysis, the medical journals in EWL released by CAS are characterized by a high of articles from China as well as a high rate of retraction.The questionnaire results showed that most burn specialty scholars believe that the establishment of such a list is beneficial to enhancing research integrity and improving journal quality management as well as for selecting journals for publication.The EWL can provide important practical data for the 'global alliance of early warning journals' in the future.
Twenty-six early warning SCI-indexed medical journals on the EWL released by CAS at the end of 2020 were assessed with data such as the publisher, database inclusion, Journal Citation Reports (JCR) quartile, CAS quartile, open access (OA) or not, article processing charge, publication cycle, 2019 impact factor (IF), 5-year (2014-2018) IF, 5-year (2014-2018) self-citation rate, 2019 self-citation rate, global total number of articles published, proportion of articles authored by Chinese scholars in the past 5 years (2016-2020), and retractions in the past 5 years (2016-2020) to analyze the characteristics of these journals and to select characteristic indicators.These data were compared by category to determine the performance of the characteristic indicators and select the main indicators.
Between 2016 and 2020, only three journals had no retractions, 22 journals each had cumulative retractions of fewer than 50 articles (1-43 articles) and one journal, the European Review for Medical and Pharmacological Sciences, had 174 retractions in total (with a peak of 96 articles in 2020), among the 26 early warning SCI indexed medical journals.In addition, journals with more than20 cumulative retractions included the International Journal of Clinical and Experimental Pathology (31 articles), the International Journal of Clinical and Experimental Medicine (21 articles), MEDICINE (43 articles, among which the peak was 13 articles retracted in 2020), Oncology Letters (41 articles, among which the peak was 15 articles retracted in 2020), Experimental and Therapeutic Medicine (25 articles), OncoTargets and Therapy (31 articles, among which the peak was 20 articles retracted in 2020), and LIFE SCIENCES (38 articles, among which the peak was 19 articles retracted in 2020).See Supplemental Table

Fig. 1
Fig. 1 Factors considered by the 424 respondents for article publication.

Figure 1
Figure 1 and Table2show that the main factors that the respondents considered for article publication included journal reputation; more than four-fifths (87.3%) of the respondents stated that EWL had an impact on their article submission choices, and almost all respondents believed that early

Fig. 2
Fig. 2 Reasons for early warning of journals perceived by the 424 respondents.

Table 1 .
Characteristics of the 26 SCI indexed medical journals at different risk levels.
CAS isChinese Academy of Sciences; The sample size of 2019 IF and self-citation rate are 26.The sample size of the 5-year (2014-2018) IF and the 5-year (2014-2018) self-citation rate are 25 during 2016 to 2020 M (P25, P75).

Table 2 .
Understanding of the policy of early warning journals by the 424 respondents.
EWL is Early Warning List.