How to Be a More Effective Reviewer

David M. Schultz Chief Editor

Search for other papers by David M. Schultz in
Current site
Google Scholar
PubMed
Close
https://orcid.org/0000-0003-1558-6975
Open access

© 2022 American Meteorological Society. For information regarding reuse of this content and general copyright information, consult the AMS Copyright Policy (www.ametsoc.org/PUBSReuseLicenses).

© 2022 American Meteorological Society. For information regarding reuse of this content and general copyright information, consult the AMS Copyright Policy (www.ametsoc.org/PUBSReuseLicenses).

Monthly Weather Review needs over a thousand peer reviews each year to maintain the high quality of articles that our readership have come to expect. We value our volunteer reviewers and recognize their investment that keeps our journal operating. As a result of their experience, professionalism, and generosity, the majority of these thousand reviews are thoughtful, thorough, and constructive. I hear about the negativity in peer review in journals from other disciplines, and I am thankful that we rarely see such serious issues in Monthly Weather Review.

We desire to maintain this character and quality of our reviews in this 150th year of publication, and so let us reflect on the process of being a reviewer. As a starting point, the American Meteorological Society (AMS) provides a list of obligations for reviewers to follow, and we encourage all reviewers and authors to read this list (AMS 2022a). AMS also offers some instructions and basic guidance for reviewers on their website (AMS 2022b) and in their reviewer-invitation letters. Other resources exist across many books and journals from different disciplines to help reviewers to understand the review process and write their reviews (e.g., chapter 20 in Schultz 2009; Lovejoy et al. 2011; Drotar et al. 2013; Caligiuri and Thomas 2013; Berk et al. 2017; Stiller-Reeve 2018; Warner 2019), and some publishers even have their own reviewer training and accreditation (e.g., Institute of Physics and Springer Nature). In this editorial, Monthly Weather Review asks each of us to rededicate ourselves to insightful, constructive, and supportive peer review, both to maintain the quality of published articles and to make all of us more effective reviewers. Below, we provide guidance to potential reviewers on the peer-review process and how to contribute most effectively to it. The remainder of this editorial has four sections: receiving the invitation, writing the review, after the initial review, and communicating with the editor.

Receiving the invitation

There are a few things to consider when first receiving the invitation to perform the review. We list them here:

  • Respond as quickly as is feasible. Monthly Weather Review prides itself on being a leader among AMS journals in the shortest time for authors to receive an initial decision. The sooner that we know whether you can accept the invitation, the speedier the process is. Letting the editor know as soon as possible is especially important if you plan to decline the review, so that editors can identify other potential reviewers sooner. It is better to let an editor know immediately that you are interested but require more time to decide than to sit on an invitation in the hopes of buying time.

  • Each manuscript (defined here as a paper submitted for peer review but not yet accepted or typeset) typically requires two or three peer reviewers. Thus, the Golden Rule of Reviewing is to review double or triple the number of manuscripts that you submit per year.

  • We all have too much to do, and Monthly Weather Review values the volunteer contributions that reviewers make. If you would like to do the review, but need more time than the typical 4 weeks, please ask the editor if an extension is possible. Even if you accept the review and then something comes up later, please e-mail the editor and ask for an extension. In most cases, an extension of a week or two is not a problem and is something the editor can easily accommodate.

  • Invitations to review should be treated confidentially. If you are invited to perform a review but cannot do it, please provide any recommendations for other potential reviewers to the editor rather than contacting those individuals directly. If you want to jointly write a review with your student as a learning experience for them, then that is great. Just ask the editor first. In most cases, this is not a problem. However, we do like to maintain the confidentiality of the peer-review process and know who has awareness of and access to the manuscript (Schultz 2019).

  • If you have reviewed a previous version of the manuscript for another journal, please let the editor know. In some cases, the editor may wish to have a fresh pair of eyes perform the review; in other cases, the editor may wish to know whether the manuscript has improved from the previous submission.

  • Reviewers may not have expertise on all aspects of a manuscript, which is why it is sometimes necessary to have a larger number of reviewers. For example, a manuscript describing the statistical analysis of a large ensemble of high-resolution convective-storm simulations generated by varying model parameterizations may require expertise on statistics, ensemble modeling, convective storms, parameterizations, and high-resolution modeling—all of which might be difficult to find in two or three reviewers. Alternatively, having a reviewer comment on the readability of the manuscript from the perspective of a nonspecialist can be valuable to helping that manuscript have greater impact. If you feel comfortable commenting on only some parts of the manuscript, just let the editor know on which sections of the manuscript you will focus.

  • If you feel like you may have a conflict of interest with any of the authors of the submission, please inform the editor to assess whether you feel that you can provide an objective review. The editor will determine whether to continue your reviewer assignment or to find another potential reviewer to avoid the appearance of a conflict of interest.

  • If this is your first review or you would like guidance in writing your review, please inform the editor. The editor can work with you and give you feedback to ensure a high-quality and effective review.

Writing the review

The average reviewer spends about 10 h on the review, with three-quarters of reviewers spending between 3.5 and 12 h (Golden and Schultz 2012). Thus, reviewing is a considerable investment of the volunteer reviewer’s time to help improve someone else’s manuscript for no formal credit. Here, we discuss how reviewers can best use their time to help improve the manuscript and help the editor to make a decision. The best reviews are those that are thorough, identify strengths as well as weaknesses, are critical yet constructive, and are respectful to the authors.

  • Nearly all manuscripts have strengths and weaknesses; well-balanced reviews should list both. Listing the strengths gives reassurance to the author and editor that the work is valid and provides evidence for why such a manuscript should be put on a track to revisions leading to publication (e.g., Martin 2008; Min 2014). Listing the weaknesses provides to the author areas that need to be improved. Such information can be provided in a short introduction to the review, which should also contain a summary of justification for the recommendation.

  • The primary focus of peer reviewers is on the scientific content of the manuscript. After all, being a subject-matter expert is why you were asked to be a reviewer. The suitability and quality of the science are generally what will determine whether the manuscript is sent back for revisions or rejected. Publishing is also about the authors’ ability to communicate their results to the readership, however. If you are able to provide detailed guidance to authors on ways to improve the presentation of their research to increase its potential impact on the field, then please do so.

  • Exhibit a clear structure and organization to your review. Sorting the comments into sections such as fatal flaws (if any), major comments, and minor comments helps to quickly identify for authors and editors the most important comments. Number comments within each section in case the authors, other reviewers, or the editor wants to cross-reference them.

  • Do unto authors as they would do unto you. Write your reviews to be empathetic to the issues faced by the authors in making revisions. When suggesting what needs to be improved, be positive, constructive, and persuasive. The goal is to get the authors to acknowledge your concern, accept your suggestion, and revise the manuscript accordingly. Harsh or unprofessional criticism is less likely to be implemented and more likely to make the author confrontational rather than open to your concerns. Practical suggestions on how to improve the submission are generally received more positively than those saying what is wrong. For example, “This paragraph would be easier to understand if…” is better than “This paragraph is poorly structured.”

  • Be specific in your comments. For example, if you feel that the manuscript is too long, provide specific locations that could be deleted or suggest strategies that the author could use to prioritize the content. In a similar vein, provide resources to help the authors to overcome their weaknesses. You can also provide good examples from other articles for the authors to emulate or use good examples from elsewhere in the authors’ own manuscript. Last, there are many useful online sources for improving writing. Here are some examples:

  • Reviewers are typically selected for their expertise pertinent to the manuscript, which means a publication record on that subject. If the authors have neglected to cite your relevant publications, it is acceptable to request that they be cited, where appropriate. However, only request citations that are relevant to the manuscript.

  • Reviewers do not necessarily need to tell authors how to address the comments to the point of suggesting the work to be done (and rising up to the level of a coauthor), but it can be useful. Distinguish between comments that must be addressed for the manuscript to be publishable, comments that would improve the manuscript but are not essential (“if the authors wanted to” or “had time”), and suggestions for future research that are not part of the present manuscript.

  • Sometimes manuscripts may be difficult to understand because they are not written well. In such cases, reviewers are under no obligation to fix the entire manuscript. A simple statement that the manuscript needs improving along with some examples of the kinds of errors that exist is all that is needed. Sometimes reviewers will write, “I’ve noticed grammatical problems, but the copy editors will fix those.” Although it is part of the copy editors’ job to fix errors that authors and reviewers miss, authors should be taking on the bulk of that effort and not AMS staff.

  • Reviewers should avoid making comments that assume the background or motivation of the authors, such as expecting manuscripts to be proofread by “native English speakers.” Such language is unacceptable in reviews at AMS journals because there is no monopoly on high-quality writing in such groups. Reviews containing such statements will be sent back to the reviewer for revision before being passed on to the author.

  • Avoid instructing authors to meet AMS style and formatting (e.g., specific referencing styles) as these may change from time to time.

  • Often the hardest recommendation to make, but the one most helpful to the editor, is for the reviewer to give a clear and evidenced statement that a manuscript should be rejected. A recommendation for rejection should not necessarily be viewed as a bad outcome because rejected manuscripts often can be improved, resubmitted, and published. A recommendation for rejection sends a strong message to the authors that a substantial reworking of the manuscript is needed. What are the criteria that would suggest a rejection recommendation?

  • In some cases, reviewers sign their reviews to let the authors know that they are willing to help them further or to receive acknowledgment. It is certainly not a requirement, and most reviewers retain their anonymity, but it is a possibility. One way to ensure that your comments remain constructive and collegial is to imagine that you would be signing your review. Not wanting to have your name associated with your own review is a good indication that you may want to consider revising your review, regardless of whether you will be signing it.

  • If you feel that you may be emotionally invested in your review, set the review aside for a day or two before submitting it. Give yourself the distance to reflect on your tone and how your language might be received by the author. Revise the review, and then submit it.

  • When submitting your review, you will have the opportunity to provide confidential feedback to the editor. This feedback is a great place to provide additional information that will help the editor to make the best decision possible for the manuscript. This feedback can include a more candid assessment of the work, a statement about teetering between two different recommendations (e.g., “major” vs “reject”), a note that the required changes may take a long time to complete, an assessment of whether authors should be encouraged to resubmit a rejected manuscript, a note about the objectivity of your review (particularly in cases of potential conflicts of interest or emotional investment), or other information that would be relevant to the editor. You can also note your level of comfort with the subject material.

  • Remember to proofread your review before submission. Using a spell checker and grammar checker will help to convey the professionalism in your review.

After the initial review

  • A reviewer who recommends major revisions or rejection is usually invited to review a revised version. We appreciate that multiple rounds of review can become tedious for reviewers, and we hope you oblige to maintain continuity throughout the review process. Nevertheless, we also recognize that your situation may change in the meantime.

  • Reading the other reviewers’ comments is an opportunity to learn from them, including how they structured their review, how they interpreted the manuscript and its findings similarly or differently than you did, and what they emphasized differently in their review relative to yours. Such reflection should only be viewed positively with the goal of learning and improving. Remember that different reviewers will bring different specialisms and experiences to their reviews, so consistency among the reviews with all making the same recommendation and identifying the same concerns is not necessarily the desired outcome of the peer-review process (Schultz 2010).

  • When reviewing a revised manuscript, there is no obligation to comment on the other reviewers’ comments, but you are welcome to provide any thoughts that you have about them and whether you feel the authors addressed those comments with the revisions.

Communicating with the editor

  • Editors value all input from reviewers, including the review comments themselves, any “messages to the editor” submitted with the review, and any direct communication. Please contact the handling editor with any questions, updates on progress, or additional thoughts that you might have about the submission. The more information that the editor has about your impressions of the manuscript, the better informed the editor’s decisions will be.

  • Feedback to editors about their decisions is also welcome. If you are particularly impressed with the decision or feel that it could have been made differently, please feel free to send this information to the editor. This kind of feedback helps us to ensure the effectiveness of the peer-review process for Monthly Weather Review.

Now, if you have been reading this editorial and you think, “Sounds like fun. How do I get involved?” then we have two ways for you to do so:

  1. you can officially register your interest at the website AMS (2022c) or

  2. if you know editors, you can e-mail them and express your interest. Let them know the types of manuscripts that you would be willing to review or that you feel you have expertise to review.

Please recognize that you may not be invited right away if no suitable manuscripts arrive. If you want to become an associate editor or editor, then the surest way of getting there is to be an effective reviewer and indicate your desire to an editor.

To raise a final point, AMS is a nonprofit society, and the AMS Publications Department is—and remains—staunchly independent. AMS Publications Department staff are salaried, but the Publications Commission, chief editors, editors, and reviewers are all volunteers, dedicated to seeing a high-quality and professional portfolio of journals in the end. Any budget surplus is reinvested back into the society and its membership. That money goes toward offsetting publication charges for authors from countries who require waivers to publish, keeping zero page charges at our journal Weather, Climate, and Society, and advancing the educational mission of the society. Currently, archives of all AMS journal articles older than one year are open access (Rauber 2017), even though most were subscription-only access when first published. AMS was one of the publishers that led the way (Schultz and Potter 2022, p. 28), and not many professional societies can boast that kind of record in making its archives of a subscription journal freely available. Furthermore, AMS journals are governed by and for the research communities that the journals serve. This business model stands in stark contrast to that of for-profit journals. Those journals also have editorial boards involved in decision-making but not to the extent the AMS journal boards are as part of a volunteer-governed society. In addition, the volunteerism of the editorial system and peer reviewers in a for-profit journal can be viewed as exploitive, with the oversized profits going to shareholders in the company. That is why I and all the editors of Monthly Weather Review thank you for your support through your volunteer effort to help maintain the quality of the manuscripts that we publish. It helps to retain our independence and the strength of our professional society.

Acknowledgments.

I thank the following individuals for comments that improved this editorial: Editors Kristen Corbosiero, Clark Evans, Jidong Gao, Josh Hacker, Ron McTaggart-Cowan, and Russ Schumacher; AMS Peer Review Support Coordinator Andrea Herbst, Senior Peer Review Support Associates Hayley Charney, Felicia Gullotta, and Erin Gumbel, Peer Review Associate Christine Ziebarth, and Peer Review Assistants Cristina Barletta and Robbie Matlock; Production Team Coordinator Liz Wright; and Publications Director Gwendolyn Whitaker. I also thank the reviewers, who provide a thousand reviews a year. Thank you for your dedication and your time.

REFERENCES

Save
All Time Past Year Past 30 Days
Abstract Views 0 0 0
Full Text Views 2108 550 212
PDF Downloads 1865 472 113