Peer-reviewed journals and the authors who publish in them owe a tremendous amount of gratitude to the reviewers who take the time to read the manuscript submissions in depth, think carefully about them, and doublecheck references, methodologies, results, and the validity of conclusions, as well as suggest ways to strengthen manuscripts in order to improve the clarity and context of the message for the benefit of readers and the scientific community. Academic Psychiatry, as a peer-reviewed journal, could not carry out its central mission without its peer reviewers, who volunteer many hours of work each year. Therefore, the task of finding capable and experienced reviewers is of paramount importance to the journal. The primary goal of this editorial is to provide guidance on peer reviewing to those who are contemplating serving as reviewers, including those comparatively new to reviewing. We also hope to benefit those who already serve as peer reviewers. Our editorial aims to specifically discuss the steps all reviewers can take to make the task of reviewing easier and more useful for authors and editors.

Several formal curricula exist on peer reviewing, but most peer reviewers are not aware of these, and peer reviewing is not taught as a routine part of residency training [1]. Editorial fellowships can include formal mentoring on how to review [2]. Recognizing the need for accomplished reviewers and considering its core mission of nurturing trainees, Academic Psychiatry launched a trainee editorial fellowship for psychiatry residents and fellows. Through a hands-on approach and with senior editorial support and encouragement, the fellowship, now in its second year, helps the trainee editorial fellows to develop as reviewers, get insight into editing processes, and write editorials. Some other journals, like the American Journal of Psychiatry, have a dedicated residents’ journal, for which residents can serve in editorial roles [3]. Most peer reviewers learn their skills without formal support, however.

Why Review?

Reviewing has several benefits. It contributes directly to the academic community in several ways, but it also offers many direct and indirect benefits to the reviewers who are volunteering their time. Engaging in a thoughtful reviewing process can increase knowledge, enhance critical thinking, allow the reviewer to stay relatively up to date with recent research, and potentially build expertise and writing skills. By watching a manuscript travel through reviews and revisions to a final acceptance or rejection (such as when reviewers can view other reviewers’ comments and authors’ responses to their comments), a reviewer gains important insights into how to write and transform a manuscript into a successfully published paper. Although every manuscript has its own journey, those authors who review manuscripts develop more insight into how to prepare and submit their own work in a format that is acceptable to potential reviewers and editors. Further, serving as a reviewer is a valued experience and skill that can contribute to promotion requirements.

Service as a peer reviewer is increasingly recognized for its value to the scientific community [4]. Journals honor and award reviewers in several ways, including granting continuing medical education credits and certificates of acknowledgment, as well as publishing a list of reviewers in an end-of-the-year edition. Some journals bestow awards for the top reviewers in an academic year. Academic Medicine has long given awards to its top reviewers. In addition to listing and thanking all its reviewers annually, Academic Psychiatry has added the category of Distinguished Reviewer to recognize those who complete a significant number of reviews for the journal in the year. Such service may pave the way for an invitation to become a member of an editorial board. In the selection of editorial board members for Academic Psychiatry, the editors pay attention to the quantity, quality, and timeliness of reviews completed for the journal.

Becoming a Reviewer

How does one become a reviewer for a journal? Once the reader has decided to embark on the noble journey of peer review, a few steps can be taken prior to receiving the first reviewer invitation. While not essential, several resources can benefit a potential reviewer before or during the process of review (Table 1) [5,6,7,8,9,10,11,12].

Table 1 Useful resources to build reviewer expertise
Full size table

It is best to start serving as a reviewer for only the journals with which one is most familiar — the journals that one uses most in academic and clinical life. (Once key journals are identified, it is a useful practice to regularly read the journals and understand their culture.) We have two reasons for this recommendation. First, the reviewer helps the journal editors decide the fit of a manuscript for the journal and its significance [13]. Thus, the potential reviewer who is also a reader of the journal can provide the reader’s perspective in helping the editors decide whether to publish a manuscript. Second, by being familiar with the articles of a particular journal, the reviewer is familiar with the different categories for submissions, as well as how successful manuscripts are structured. This familiarity makes it easier to provide constructive feedback relevant to the journal of interest. Common Academic Psychiatry manuscript types and their requirements are listed in Table 2 and in the journal’s instructions for authors [14].

Table 2 Common types of manuscripts published in Academic Psychiatry [14]
Full size table

The process by which a journal decides to invite a particular individual to peer review varies. Some journals may require publication within the journal before consideration as a reviewer. Other journals may offer a way to express interest in reviewing. One useful way to communicate one’s willingness to serve as a reviewer is by creating a user profile within the journal’s manuscript processing system and choosing areas of expertise, typically from one to three topics. For psychiatrists who are at the beginning of their careers and might not have fully developed areas of expertise, topics of interest can be selected. This process helps journal staff match submitted manuscripts with potential reviewers. Journal staff often attend professional conferences as an opportunity to connect with authors and readers, and one can use such opportunities to sign up as a potential reviewer. Academic Psychiatry also offers a “Meet the Editor” online opportunity several times throughout the year to connect with members of its sponsoring organizations. Finally, interested individuals can directly contact the journal to indicate their desire to review [15]. Simply indicating a willingness to review does not guarantee an invitation to review. Regardless, expressing interest, being familiar with the journal, and, if possible, developing a publication track record, especially in the journal of interest, can help.

Responding to the Review Invitation

The first invitation to review can be an exciting event. It is, after all, an indication of the journal’s interest in the reviewer’s expertise. In general, it is best to respond as soon as possible by accepting or declining the invitation. A quick response, even if negative, is helpful to the editors, as they can then invite another reviewer. The longer a reviewer takes to decline, the longer an anxious author waits to know the decision regarding the manuscript. It is important to look at one’s schedule and plan out the amount of time needed to submit the review in a timely fashion. If a reviewer is declining an invitation but would like to review in the future, it is important to let the editors know. Some journals ask those who decline if they might recommend another reviewer for the topic, which may be an excellent opportunity for one to suggest a colleague who is looking to become involved as a reviewer.

Acceptance of an invitation to review indicates that the reviewer has the expertise and ability to submit the review on time. In addition, acceptance indicates that reviewers believe that they will be able to review the manuscript ethically and that they do not have a conflict of interest. Some journals specifically ask reviewers to declare that they have no conflict of interest, but even when not asked, reviewers should be careful to recognize any conflicts that might exist and to declare them. Examples include having a strong bias for or against publication of the manuscript either due to the subject of the manuscript or because of an invested relationship (e.g., as a mentor or collaborator) with any of the authors [16]. Any potential bias concerning a manuscript should be identified, disclosed, and managed so that those biases are not influential in the recommendations provided to editors.

Nuts and Bolts of a Review

One way to get started with the review is to do a quick initial read of the manuscript, to get the general scope of the paper. During this read, the reviewer should consider if the topic of the manuscript is relevant to the mission of the journal, if it is potentially interesting to the readers of the journal, and if the manuscript adds to the literature that already exists on the topic. Some reviewers prefer to set the manuscript aside after the first read and come back to it a day or two later. It may be helpful for early-career reviewers to consult with a mentor or supervisor, to discuss the topic in general, and to get some guidelines on issues they might want or need to consider further. Nevertheless, the second reading should be more detailed and focused. It is useful to do this reading section by section, while taking notes that will help in the final compilation of the submitted review.

The title and abstract are two parts of a manuscript that the reviewer should consider before and after carefully reading the complete manuscript. The title should clearly indicate what the reader can expect from the manuscript. The abstract, and of course the manuscript, should provide what the title promises. As the abstract is a short summary of the manuscript, it should provide the reader with a complete overview of the manuscript, especially the important findings and takeaway message. The language of the abstract must be as concise as possible, with no unnecessary information, given the abstract’s central role in summarizing the paper for those readers who might not critically appraise the manuscript itself.

The introductory section should clearly set the stage for the rest of the manuscript. The purpose of this opening text is not to provide a comprehensive review of the literature but to introduce an issue or problem, indicate its importance (supported by references), report on how methods or solutions to date may have fallen short (supported by references), and state how the authors will aim to rectify any deficiencies or improve on the available literature. This aim may be fulfilled by replicating previous findings when justified. The specific goals, objectives, or any hypotheses are stated in the introduction, including a statement about how readers might benefit from the information presented in the manuscript.

While the introduction does not need to be an exhaustive review of the literature, it should have references that are up to date. It is easier for reviewers to judge the adequacy of this section if they already have expertise on the topic. For the reviewer who is still building expertise, this section can be a gold mine of information, providing references to literature with which the reviewer can become familiar. In short, the introduction is a crisp, concise overview of the topic, setting the stage for the findings, arguments, and limitations considered in the rest of the manuscript. For reviewers who are deepening their expertise, some time will go into reviewing the relevant literature to confirm that the authors have used the best available literature appropriately, that those references were valid to the statements made by the authors, that the references were the primary sources of the statements, and that the reference section does not omit important or relevant information.

The methods and results sections are the core of empirical manuscripts. If these sections are strong, deficits in other areas can be more easily corrected. The methods should directly follow the goals and objectives stated in the introduction and should be described adequately enough to allow for replication of the study and a thorough understanding of the strengths and weaknesses of the findings and conclusions [17].

Some questions to consider, particularly for quantitative or qualitative studies, include whether the authors have mentioned an ethics review (e.g., institutional review board approval or exemption) and, perhaps, whether recruitment of participants required an informed consent process free of coercion; whether the research design and implementation was clearly described with all requisite details to enable a judgment of its adequacy to answer the question or questions that were posed; whether numbers and response rates, including dropout rates, were provided when applicable; whether information on the validity and reliability of the outcome measures were provided; whether the statistical analyses were appropriate and sufficiently well described; and whether statistical power was considered and adjustments undertaken to protect against the possibility of confounding and multiple variables.

If the statistical analyses are too complex for the reviewer, it is useful to add a comment to the editors about this difficulty in the submitted review. In such cases, editors can ask for an additional review from a statistical expert.

The results should be presented in a clear manner, such that the reader can clearly understand the significant findings. While reviewing the text, it is important to also review the tables and figures that accompany these sections. Tables can help to highlight the findings which most deserve to stand out and can assist readers in their understanding of what are the most important findings. Substantial overlap or repetition should not occur, however, between information in the tables and text.

For educational case reports [11], the methods section is replaced by a description of the educational intervention and the details of its implementation. The results section is replaced by a description of the outcomes and the lessons learned. For such case reports, the reviewer evaluates the rigor of the authors’ approach in developing and evaluating the educational advance (such that it merits publication in a scientific journal); the degree to which common curricular challenges are addressed (such that the case is of interest beyond the authors’ institution); and the practical applicability and sustainability of the intervention. As noted earlier, it is helpful for reviewers, seeking to provide the best advice to authors, to be familiar with how educational case reports, as well as other manuscript submission categories, are structured [14].

In the discussion section, the reviewer should look for a thoughtful consideration of the context and practical implications of the authors’ findings. Some questions reviewers may address include whether the findings sufficiently addressed the specific goals and objectives of the manuscript; whether the findings were presented in a balanced way and without embellishments or an undue positive bias; whether all important limitations were clearly identified; and whether additional methods or research might help to clarify or supplement the findings, arguments, or recommendations of the paper.

Stepping back to look again at the manuscript after completing a detailed reading can further assist in a judgment of the overall manuscript. This step includes a critical rethinking of the priority of the paper, the adequacy and validity of the methods, and the importance of the results. It is also helpful for authors to hear from reviewers what they liked about the manuscript. Authors will have, no doubt, put a lot of work into their submission, and they should be admired for their efforts. Reviews that are unduly negative or pejorative will damage authors’ enthusiasm and therefore are to be avoided. As noted earlier, reviewers should be very careful not to let negative or positive biases undermine their judgments and counsel in the review process.

Writing the Review

While the detailed notes for each section are useful, the submitted review should be a clear and concise summary and not simply a compilation of notes or a checklist. The notes are the scientific backbone of a review, while the final written review can be considered an art. For a novice reviewer, it can be useful to read reviews others have written. Some published manuscripts share examples of useful and not-so-useful reviewer comments [8, 18].

The review’s introductory summary statement should discuss the overall quality of the manuscript, the value of the topic, major strengths of the paper, and any major flaws in the goals or methods sections that can impact the acceptance decision. Positive remarks can be included in the introductory comments and can be followed by a listing of specific concerns that authors should address to improve the manuscript or to meet publication standards. If the reviewer has many critical comments, it can be useful to divide them into major and minor comments. Major comments are usually critical flaws that need to be addressed to make the manuscript meaningful. Minor comments can be smaller edits related to improving the clarity and usefulness of the manuscript. Specific feedback is useful when listed as points relevant to each section. It is not useful to share notes that state, for example, the “introduction is fine” and the “abstract communicates what it should.”

Most or all of the reviewer’s comments should be directed to the authors. The critical feedback should be written in language that is encouraging and kind. Wherever possible, the reviewer should be highly specific in the comments, to serve as a guide for revising a particular passage or section of the manuscript. While reviewers do help in the acceptance and rejection of manuscripts, the spirit of a review is similar to the formative feedback supervisors give trainees. Academic Psychiatry, in particular, takes a developmental attitude and wants to help authors improve. In short, a good review in the tradition of Academic Psychiatry should provide an educational experience for the authors. The overall mission is to help the authors improve their critical thinking and writing skills for this and future manuscripts. The comments from reviewers to the authors should not include any remarks related to whether the manuscript should be accepted or rejected, as the publication decision is made by the editors, not the reviewers.

It is not always necessary for reviewers to include comments directed to the editor. This section can have brief additional, confidential comments providing clarification of the reviewer’s recommendations. Comments made in this section should not be discordant with what is written in the section for the authors. For a review to be effective, the majority of the feedback should be shareable with the authors, and confidential comments for the editor should be more the exception rather than the rule. It can be an educational process for a novice reviewer to have an experienced colleague read the review before submission, as long as the reviewer maintains the manuscript’s confidentiality.

Once a review is completely written, the reviewer has the final task of making a decision recommendation to the editor. For Academic Psychiatry, a reviewer can make the following recommendations: Reject (indicates the reviewer does not believe that even substantial revisions would make the manuscript acceptable); Major Revision (indicates the reviewer believes that the manuscript has major issues but that the authors have potential to address them); Minor Revision (indicates the reviewer believes that the manuscript has only some minor issues which would be relatively easy for the authors to address); Accept (indicates the reviewer believes that the manuscript does not need any edits at all). While most journals, including Academic Psychiatry, will ask the reviewer for a recommendation of whether the manuscript should be rejected, accepted, or revised, the final decision is made by the editors. Because different reviewers may have differing decision recommendations for the same manuscript, it is the detailed content of the reviews, rather than the decision recommendation itself, that most helps the editors make their final decision.

After Submitting the Review

Once all the reviewers have submitted their reviews and recommendations, the editors make the final decision and add comments as needed. For manuscripts that receive a decision of major revision, the journal editors usually invite the original peer reviewers to complete an additional review. In the documentation accompanying the revised manuscript, authors are expected to address each reviewer comment and explain the changes made or explain why any of the suggested changes were not made. In the circumstance of conflicting reviews, the editorial comments guide the authors on which comments do not need to be addressed or on how to balance those comments. A flow chart detailing the editorial and publication process for Academic Psychiatry can be found in a previously published column [19]. The decision letter that is sent to the authors is shared with the reviewers. Reading the decision letter carefully can be an educational experience for the reviewer, as it provides insight into what the other reviewers and editors thought about the manuscript. It is essentially informal feedback to reviewers on what they might have missed or could potentially pay attention to the next time they review.

It is very satisfying to read a published article to which the reader contributed as a reviewer. At this stage, the reviewer can simply sit back and enjoy!