Research and Discussion Note
Peer review process in medical research publications: Language and content comments

https://doi.org/10.1016/j.esp.2009.07.002Get rights and content

Abstract

Every article sent by an author to a reputed scientific journal undergoes a rigorous editorial evaluation. The editor has the final responsibility of accepting or rejecting manuscripts and thus can confer authority and validity on the author’s research and help to disseminate new knowledge. In this task, editors make use of a panel of expert peer reviewers in the field who examine the manuscript and make recommendations. Some aspects of the peer review process have been investigated by medical scientists and by linguists but to the best of our knowledge, there have been no studies conducted on peer reviewer comments of medical articles written in English by Italian researchers.

The present study aimed to establish the most frequent types of comments made by peer reviewers to identify the most frequent linguistic problems that Italian researchers encounter in this process. A collection of clinical research manuscripts submitted by Italian medical researchers to reputable English language journals were analysed together with the comments by editors and reviewers.

The most frequent comments and criticisms were mainly in the area of scientific and methodological content, followed by lexical and grammatical errors, clarity and verbosity or repetition. An awareness of the features which might affect the acceptance or rejection of manuscripts may help novice writers and furnish training materials to aid researchers in writing publications in English.

Introduction

There are roughly 1 million new scientific publications each year, according to the numbers of abstracts published in the scientific abstracting database of the US National Library of Medicine in Bethesda, but because many manuscripts are rejected, editors of scientific journals play a significant role as gatekeepers in the publication quality and trends in scientific research. The editor has the final responsibility of accepting or rejecting manuscripts and can confer authority and validity on the author’s research and help to disseminate new knowledge. In this task, editors make use of a panel of expert reviewers in the field called peer reviewers, who examine the manuscript and can recommend rejection, acceptance of the article in the version submitted or can send it back to the author enclosing their comments/criticisms, often recommending changes to be made in the text. Each manuscript is usually reviewed by two or three reviewers, who may at times express conflicting opinions. Authors often accept the reviewers’ suggestions which may be useful in improving the text.

In the highly competitive arena of academic publications, authors are particularly concerned about the international prestige of the journals in which they publish, in order to ensure high visibility and reader audience. The visibility of scientific journals is given by an index called the Impact Factor (IF), calculated using reader access as well as citation frequencies. It is well known that the prestige journals in medicine are published in English and therefore, it is logical that researchers who are non-native speakers of English (NNSE) might feel disadvantaged and marginalized in such competition because they are often unaware of the linguistic conventions and strategies commonly used in English language journals (Belcher, 2007, Canagarajah, 1996, Gupta et al., 2006, Loonen et al., 2005, Myers, 1990, Swales, 1990, Swales, 1996 and more recently Giannone, 2008).

The fairness of the review process in science manuscripts has been addressed by Berkenkotter and Huckin (1999, p. 62) and they suggest that “although peer review is not infallible, it remains the primary means through which authority and authenticity are conferred upon scientific and scholarly papers by journal editors and the expert judges whom they have consulted”. They describe peer review comments in terms of speech act theory as an argumentative discussion of author and reviewer. Bias by peer reviewers may be introduced by way of a negative authorial voice, that is, opinion constructed by the reviewer about the author (Matsuda & Tardy, 2007), but though asymmetric in discourse terms, peer review does offer guarantees of fairness like blinding, such that the reviewer does not know the name or status of the author and can thus objectively judge the quality of the publication, authorship and conflict of interest (Callaham, 2002). Beyer, Chanove, and Fox (1995) studied peer review comments of Management manuscripts and advocated a coaching style of reviewer comments to improve publication outcomes.

Peer review reports have long been considered an occluded genre (Gosden, 2001, Swales, 1996) and this has made it difficult to study their effect on publication outcomes. In order to improve publication rates, medical editors have published a content-based checklist of study design items under the name of the CONSORT1 Statement, (Begg et al., 1996). In a conference dedicated to Medical Journals Editing (2002) and reported in a special issue of the Journal of the American Medical Association (JAMA), particular attention was paid to creating checklists concerning the soundness of the scientific methodology of research manuscripts (Day et al., 2002, Schriger et al., 2002) but these checklists were not data-driven or pertinent to any pool of manuscripts but rather derived from a generalized reviewer opinion (Schriger et al., 2002) or from a meta-analytic overview (Jefferson, Alderson, Wager, & Davidoff, 2002) and none of the checklists was concerned with the special linguistic problems NNS scientists faced.

In the area of Applied Linguistics for EAP (English for Academic Purposes), Gosden (1995) proposed a social-constructionist interpretation of reviewer comments in order to identify communicative strategies such as improvements in moves or “rhetorical machining” and suggested wider use of context frames such as connectors by non-native speakers of English (NNSE) who had unsuccessfully submitted research articles to international journals. He stressed the need for “cultural awareness” (p. 4) on the part of young research authors in handling the social interaction of scientific research communication. Similarly, Kourilovà (1996) analysed a corpus of peer review reports on rejected medical manuscripts of Slovak scientists to identify the pragmatic communicative functions of reviewer comments. She suggested that Slovak writers of English scientific discourse need to master pragmatic skills and socio-cultural strategies. The importance of authors being able to recognize the difference between social interactional (or interpersonal) as opposed to technical comments by peer reviewers was also noted by Gosden (2001). Gosden (2003) also examined a corpus of 40 manuscript submissions with some of their reviewer comments to identify the frequencies of comment types and the motivations behind reviews in order to help novice scientists in revising their manuscripts. In this study, Gosden noted that about two-thirds of all peer comments were interactional in nature and identified the sections of research articles (RAs) requiring changes as mainly in the discussion, technical details, author claims and reference to others’ work, and a category called ‘format comments’. The presence of ‘language-use comments’ was identified but not classified.

Recently, Belcher (2007) studied comments in a corpus of 29 linguistics research papers comparing network and off-network (periphery, EL, EIL or NNSE) scholars. She identified reviewer comments pertaining to different sections of RAs such as the methods or discussion section, but did not investigate frequencies of commentary types. She observed that the provenance of the author was not as important as author willingness to revise-and-resubmit a manuscript and noted that both networked and off-network authors shared the “same types of content and expression problems” (Belcher, 2007, p. 18). Like Gosden, Belcher described the category of comments on “language use” as frequent, and confirmed that although this category received much attention, the composition of this category of language use and style was “amorphous” and remarked that “it is easy to see why some authors might feel that language stands in the way of publication” although she notes that no paper had been rejected for linguistic reasons (Belcher, 2007, p. 7). Belcher observed that authors need to recognize that requests for change may be negotiable, echoing the observations of Gosden (2003) and Kourilovà (1996).

To the best of our knowledge, no study has been carried out on reviewer comments about medical manuscripts prepared by medical scientists in Italy, who might be considered off-network and NNSE researchers. This paper aims to extend the work of the scholars mentioned in the previous paragraphs, but it also incorporates categories proposed by medical journal publications on peer review. We attempted to identify the most frequent comments and classify them. We therefore collected a corpus of reviewer comments on manuscripts submitted by Italian scientists to international journals and used the categories of Gosden (2003) as a base to classify our corpus of reviewer comments on Italian medical manuscripts in an attempt to find answers to the following queries:

  • What kinds of comments are made?

  • Which comments were more common – linguistic or scientific-methodological criticisms?

Section snippets

Selection, processing and data-handling of the corpus

We solicited reviewer comments together with manuscripts in various stages of review from colleagues, who were carrying out research in our medical school. We collected reviewers’ comments from 17 manuscripts submitted to English language medico-scientific journals by NNSE researchers working in Italy. I was unknown if the reviewers were central or periphery scientists.

All the publications selected were research articles (RAs). Other publication types such as reviews or letters to the editor

Publication outcomes of the manuscripts presented

All 17 manuscripts submitted were research articles as defined by Swales (1990). The medical fields comprised neurology, dermatology, internal medicine and surgery.

On the first round of revision, 15 of the 17 manuscripts (88%) were returned for revision and resubmission, but one article (6%) was accepted immediately, with no requests for linguistic revision and only minimal modifications of the report on technical aspects of the study and another manuscript was rejected because of unsuitability

Discussion

Our data confirm the opinions of others such as Sionis (1995) on French scientists, Mišak, Marusìć and Marusìć (2005) on Croatian medical manuscripts, Kourilovà (1996) on Slovak manuscripts, and Gosden (2001) that NNSE appear to have both language and scientific methodology problems when they prepare manuscripts for international journals. Which of the two is the predominant reason for rejection is difficult to establish and we do not wish to suggest causality between reviewer comments and

References (33)

  • C. Sionis

    Communication strategies in the writing of scientific research articles by non-native users of English

    English for Specific Purposes

    (1995)
  • C.B. Begg et al.

    Improving the quality of reporting of randomized controlled trials: The CONSORT Statement. (consolidated standards for the reporting trials)

    JAMA Journal of the American Medical Association

    (1996)
  • C. Berkenkotter et al.

    Genre knowledge in disciplinary communities

    (1999)
  • J.M. Beyer et al.

    The review process and the fates of manuscripts submitted to AMJ

    Academy of Management Journal

    (1995)
  • G. Bordage

    Reasons why reviewers reject and accept manuscripts: The strengths and weaknesses in medical education manuscripts

    Academic Medicine

    (2001)
  • P. Brown et al.

    Politeness: Some universals in language use

    (1987)
  • Cited by (71)

    View all citing articles on Scopus
    View full text