Abstract
Peer review is essential to the quality of scientific publications. As the volume of submissions to these journals increases, the need for an expanding cadre of trained reviewers also increases. Many reviewers do not receive formal training on conducting a peer review or education on the peer review process, however. The author presents an overview of the peer review process and provides guidance for conducting high-quality reviews of manuscripts submitted to The Journal of the American Osteopathic Association and other scientific journals.
It is the mission of The Journal of the American Osteopathic Association (JAOA) “to advance medicine through the timely publication of peer-reviewed osteopathic medical research.” Accompanying the mission statement are several goals including, “to ensure the professionalism of the publication process.”
To achieve its mission and goals, The Journal must rely on its volunteer peer reviewers. According to former
JAOA Associate Editor Felix J. Rogers, DO, “the
JAOA requires an ever-expanding pool of peer reviewers committed to the prompt and comprehensive evaluation of each manuscript.”
1
All scientific journals, including the
JAOA, are constantly recruiting new peer reviewers. But, many people who are being called on to conduct a peer review may be unfamiliar with the requirements and expectations of the process.
2 Most physicians, scientists, and other experts who volunteer their time to review receive little training—formal or informal—in the critical review of research articles or in the peer review process.
3 Black et al
4 suggest that to ensure the quality of peer reviews, journals should train their reviewers.
In the present article, I provide an overview of the peer review process and describe in detail the general and specific tasks required of peer reviewers.
According to the International Committee of Medical Journal Editors,
5 “peer review is the critical assessment of manuscripts submitted to journals by experts who are not part of the editorial staff.” Evidence of the early use of peer review has been found in ancient Greek writings.
6 Prepublication review of scientific papers by experts began in the 17th century, but it was not until the 1940s that peer review became standard journal policy.
7 Today, the peer review process is a standard part of research publication: As the established method of validating one's work, it is essential to the continued growth of knowledge.
6
By definition, peer reviewers are peers of the authors.
8 The Merriam-Webster Dictionary defines peer as “one that is of equal standing with another.”
9 When selecting peer reviewers for a manuscript, editors typically choose reviewers who have demonstrated expertise in the subject. Reviewers with a background or facility in statistics may also be selected if a study's methods and results require sophisticated evaluation.
Arguably, the most important characteristic of a good peer reviewer is to be an advocate for the author. Benos et al
10 put it best when they wrote, “The most important rule is to follow the golden rule: treat all manuscripts in the same manner that you would want your own treated.”
Black et al
4 found little association between reviewer characteristics and the quality of reviews they produced. The authors did find that training in epidemiology or statistics was statistically significantly associated with higher quality reviews. Younger age and more time spent on a review (up to 3 hours) were also associated with better reviews.
There are several reasons why reviewers should volunteer their time. For one, being selected as a reviewer is an honor. The selection signifies an acknowledgment of one's expertise in the field and of one's standing in his or her career. In addition, peer reviewing benefits the profession. The reviewer contributes to the accumulated body of knowledge and facilitates the distribution of new information to our colleagues and to the public. Finally, volunteering as a peer reviewer can help advance one's career. Reviewers can list the experience on their curriculum vitae, and many journals—including the JAOA—offer continuing medical education credits for completed reviews.
Most scientific journals, including the JAOA, require reviewers to complete peer reviews using a Web-based manuscript tracking system. When a manuscript is submitted to a journal, it is first reviewed by the editor in chief and associate editors, who assign the manuscript to 2 or more peer reviewers.
Review requests are typically sent by means of e-mail. If reviewers accept a peer review request, they are given a timeframe for reviewing the manuscript. Peer reviewers of the JAOA are asked to complete reviews within 15 days. If reviewers decline a peer review request, the editors of the journal appreciate recommendations for alternative reviewers.
After completing a review of a manuscript, peer reviewers submit comments and recommendations for both the author and the editor. The editor in chief will then consider comments from all reviewers and accept the manuscript for publication without revision, accept the manuscript pending minor or major revisions, or reject the manuscript. In the event that the manuscript is accepted pending revision, authors are asked to address reviewer comments. When the authors submit a revised manuscript, the revised version is usually sent to the original peer reviewers for a second review.
At the end of the peer review process, the editor in chief and associate editors rate reviewers on the basis of the quality of their reviews and responsiveness. A reviewer's collective rating will often determine whether he or she is asked to review a future manuscript for that journal.
Journals typically use 1 of 2 peer review models: single blind or double blind. The single-blind review model is easily the most common and is the model used by the JAOA. In the single-blind review process, the identity of the reviewer is concealed, but the identities of the authors are not. An advantage of the single-blind review process is that the reviewer is able to identify and assess previous work by the authors and determine whether advances from previous work are reported. A disadvantage of the single-blind review process is the potential for the anonymous reviewer to be biased—positively or negatively—in the review.
In the double-blind model of peer review, the identities of both the reviewer and the authors are concealed. An advantage of the double-blind review process is that potential for reviewer bias on the basis of authors' previous work or institution prestige is lessened. The disadvantage of this model is that truly blind reviews are uncommon—authors frequently refer to their prior publications in a manuscript, and reviewers can easily search for authors' previous work in the subject.
11
Additional pros and cons of each peer review model are discussed by Keenum and Shubrook.
8 It is clear that no peer review process is perfect. Although peer reviewers have no control over the peer review process used by journals, they should ensure every review they conduct is fair, thorough, and unbiased.
Several excellent references are helpful in guiding peer reviewers in the review process.
2,3,8,10,12,13 In this section, I describe how I approach a peer review and include considerations for each section type. Of note, I focus the majority of my recommendations on the review of original research articles, so some points may not be applicable to all article types. Before beginning a peer review, it is important to familiarize yourself with the guidelines of the journal for which you are reviewing.
Before I agree to review a manuscript, I ask myself several questions:
-
Do I have time to dedicate to the review, considering a typical review will require approximately 3 hours?
-
Am I qualified to review this particular manuscript?
-
Have I any expertise in the subject area?
-
Do I have a conflict of interest (eg, Do I have a financial interest in the research? Have I collaborated with the author[s] in the past?)? When I believe I may have a conflict of interest, I discuss them with the editor before agreeing to review the manuscript.
After agreeing to review a manuscript and before I begin the review, I remind myself of my responsibilities as a peer reviewer. Paramount in my mind is my responsibility both to the author and to the editor. To the author, I will be an advocate. I will be collegial and offer constructive criticism. I will maintain confidentiality and keep in mind that the manuscript belongs to the author.
To the editor, I will provide my best, unbiased evaluation on the basis of my opinion of the validity of the author's conclusions. For additional general guidelines, see
Figure 1.
During my first reading of a manuscript, I focus on “big picture” issues. I consider the importance and significance of the research question and the originality of the research. Is this topic timely and appropriate for the journal's audience? Has the research question been posed before? If so, is the manuscript adding anything new to the literature?
I also look at the writing style and structure of the manuscript. Will the reader who is unfamiliar with the topic be able to clearly understand the research? Are ethical considerations (eg, financial disclosures, patient confidentiality) addressed?
I make marginal notes if I find a problem with the presentation or science of the manuscript. After completing the first read, I try to summarize the paper in a few sentences or a short paragraph. Many readers quickly skim articles; by briefly summarizing the manuscript, I can get a sense of the impression readers will get. I do not return to a detailed and comprehensive second reading until a day or so has passed.
When reading the paper a second time, I consider each section carefully and develop specific comments and recommendations for the author and the editor (
Figure 2).
It is important to read the manuscript with a critical and creative eye. In addition to assessing the paper's logic and assumptions, reviewers should look for possible improvements to clarify the presentation. It is essential, of course, to provide specific suggestions to the author. For example, when a manuscript contains references to older, outdated studies, I will not simply state, “References are outdated.” Rather, I identify more recent studies for the authors to consider citing. Of note, I focus my comments on the content of the manuscript. If I find an otherwise valuable paper difficult to read because of writing style, grammatical errors, or labored syntax, I will suggest that the authors seek writing assistance. It is not necessary for reviewers to detail grammatical errors.
Volunteering as a peer reviewer can be a valuable and rewarding experience. By following basic guidelines, reviewers can ensure their comments are professional, thorough, and helpful and improve the quality of scientific publications.