Should you review?

Remember that there is always a cost in terms of your time – to anything that you do! The more you advance in your career (and the better you do), the more opportunities you will have. So begin to practice the art of evaluating whether any given activity is worth it.

If your advisor has asked you to assist with a review, chances are that your advisor has determined that this is the right time in your professional development and that the subject matter is a good fit. In other words, your advisor has already done the “worth it” calculation for you. Additionally, your advisor will assist you. Go for it!

What if the journal contacted you directly? Congratulations! This will most likely happen if you have a publication on a similar topic, and guess what – this means you are an expert! Truly – expertise always comes in a niche area and always in increments. That is, you will never know everything, even in a given area. Likely you are now minimizing your expertise to yourself. Don’t! Practice your confidence and wearing your new “expert” hat. It will feel a bit uncomfortable at first but feels better and better as you wear it more.

How do you determine the “worth it”? The gain will be in putting this on your CV and in the insight you will gain in terms of what makes an effective journal article. You will also likely learn something about the subject matter and/or ongoing research in your field. These are all pretty big benefits. The downside is in your time, and making sure that you can do an adequate job. Here are some things to consider –

• Is the journal indexed where it should be? For public health, check PubMed. If the journal has a lot of listed publications, you are a go. If the journal is not indexed, I recommend not reviewing. It likely indicates that you have been contacted by a non-reputable journal.
• What is the impact factor (an indication of journal quality) like? (You should know this term and how to find this info. https://en.wikipedia.org/wiki/Impact_factor. Finding impact factors can be super-easy for some journals: search on google and it will be displayed. But a thorough search for official published impact factors requires having paywall access to “Journal Citation Reports”, such as through the Web of Science. Higher impact factors are better. I’m not sure how high they range, but as I look Environmental Health Perspectives is about 9 and JAMA is about 47). Lower impact factors may mean that it is a new journal or a journal whose articles aren’t cited as much. It’s OK to review for a journal with a low impact factor at this stage of your career, but it is good to at least know one (imperfect) metric of the journal quality.
• Is the topic of continued interest to you? If you have any interest in the topic, review. If the topic reflects an old interest of yours that is truly distinct from your current interests, you may want to skip this, unless you are really wanting to beef up your CV.
• Do you have the time? A first review will take some time, both in reading, thinking, and writing, and in waiting for assistance. The time could be about that for a class paper, but of course will vary. The journal will often ask that you do the review in 2 weeks. It may be possible to ask for an extension (such as an additional 2 weeks), and it is OK to ask. As always, don’t do this if it will push you over the stress edge or if it will delay other work in a way that will then greatly disappoint you later.
• Have you completed any article critiques (such as in a classroom setting?) Article critiques are a good foundation to doing peer review and if you have not done any, you will need substantial support from someone (e.g. your advisor) in doing this review. If you can not obtain that support, don’t review.
• Are you able to do a quality review? It is ultimately up to you to assess whether you can go it alone or whether you need support. Consider the role (see below) and that it is not fair to the authors to review when your ability to do so is marginal. Talk to your advisor about whether you should go it alone. Here are some guidelines: If you are a post-doc or faculty member, go it alone. If you are a master’s student or doctoral student doing coursework, get support. If you are a doctoral student in dissertation phase and have published, you may be able to go it alone or may want support. If you need the support and your advisor cannot do this, and cannot recommend someone else who is available to do this, say no.

A Cheeky Confidential on Confidentiality and Conflicts

You have likely already been given access to confidential writing – the abstract sent in the invitation to review. And if you review you will see work in progress, maybe even with the full names of all authors. Later you may see the editor letter and reviews of others if you are included in the communication to the authors following the editor decision. So much good stuff to learn from!

So – will some of this influence your work? Likely, because human nature is to constantly learn and integrate all that we learn. This is unavoidable and honestly a perk of reviewing. But will you tell a colleague about this manuscript under review at any point until it is published? No. Will you save the pdf and use some of the sentences in your own work? Of course not! Will you contact the authors and/or tell others that you had a role in reviewing? Nope.

What about conflicts of interest? You may be the right person to review even if you know the author or have similar projects underway, because you are likely highly qualified in the subject matter and methods. But of course be on the lookout for whether any emotions may be there – feelings of competition, a personal grudge, or even the desire to help out a friend. If you have any hesitation about whether these feelings cannot be kept in check, it’s best to decline to review. (Reminder: this could be a career-influencing decision you help to make and ultimately the science and the public are best served by impartiality.)

You must use your strong inner compass about respecting others to sort out the nuances of confidentiality and conflicts. Seek advice from your advisor or others with experience in these areas whenever you are unsure.

Your Role as a Reviewer

Your role is ultimately to further the published science to better the public’s health and the world!

It is your job (together with the editor and other reviewers) to prevent the publication of junk. Junk = science that is flawed and/or conclusions that are not supported. Keep in mind that once something is published, it will obtain the sheen of truth, and will be cited, included in meta-analyses, and could be used to intervene upon, set policy, or otherwise produce real-world consequences.

Another role of a reviewer is akin to an unacknowledged co-author. You will be providing constructive feedback to improve the work and the clarity of the writing. Your comments may even have a mentoring role in helping the authors be better researchers and/or writers.

Choices: Accept, Revise, Reject

You will give a recommendation to the editor on whether an article should be accepted as is, considered again after revisions (some journals ask whether you think the requested revisions are major or minor), or whether to reject.

Accept = Accepted as is.
Rare. Very rare.

R&R = Revise and Resubmit. Considered again following revisions.
Common. Very common.

Remember that the editor and journal have already pre-screened the manuscript to assure that it is appropriate for the journal and deserves the burden they will put upon their reviewers. They may make mistakes, but they are not sending out everything that comes their way.

Also – because it is very likely that you will recommend a R&R, you will want to provide detailed comments to the authors on where and how to improve. Your specific recommendations will be taken seriously by the editor and the authors. In some ways your recommendations will form a “soft contract”, where it is understood that if the authors make the requested revisions and/or provide sufficient justification for their approach, it is likely that their manuscript will be accepted and published.

Reject
Use this category when the question that can be answered with the data and methods is not a sufficient contribution, when there is a fatal flaw, or when the writing is such a mess that you cannot make heads or tails of what was done.

What is a fatal flaw?
A fatal flaw is something that prevents the manuscript from being a contribution at all. In other words, the body of literature will be more correct (closer to truth) without this publication at all. Fatal flaws can be in the study design, measurement, statistical approach, logic of the conclusions, or in any aspect of the manuscript, but they are always so large that “the answer” provided is most likely wrong. Remember, when evaluating the degree of flaws and biases, there is a continuum of types of errors from minor to major and that no study is perfect. But also remember that once something is published that it has the sheen of truth. Glossing up a wrong result with the polish of publication can lead to real-world hardship. You have signed up to prevent this from happening to the best of your ability.

Who are you writing for?

1. The editor. Your primary intended reader is the editor – the person given the power to decide whether this journal will publish this article. The editor is looking to you as an expert who will give the time to advise on:
• Whether this question is important (a worthy contribution given the previous existing literature)
• Whether the question is answered with the data and methods
• Whether the study design and methods are appropriate (without major flaws or bias. Remember that no study is perfect).
• Whether the conclusions are appropriate to the question and results

Yet note – even though your primary reader is the editor and your primary role is to help advise on whether to publish this article as-is or following revisions –
You will not tell the authors whether you recommend an accept, revision, or reject, because it is ultimately up to the editor to make this decision based on your advice and that of the other reviewers.

And most of your writing will be penned to the authors, because this is where the detail is needed, and the editor is able to read your notes to the authors.

2. The authors. Your second intended readers(s) are the author(s). You will be informing the author(s) about how you view the contribution of the manuscript – the question and methodological strengths, and of course importantly, how it could be improved.

Brief Outline of a Review

Dear Editor,

Comments to the editor should be brief (1 paragraph) to provide punchy overview. The authors will not see this part and so you may also use this section to speak candidly (but professionally of course).

Thank you for the opportunity to review. This manuscript is a contribution (or not) because (explain). Major strengths are (summarize – can be the same summary used for authors). (Next state briefly why you made the recommendation you did, including perhaps the 1 or 2 biggest recommendations to improve and why).

(Also include here anything you don’t want the authors to see. This could be areas where you did not feel you could sufficiently review. Note that you are not expected to be expert on every aspect of the approach. You may also want to comment on the quality of the writing, and/or if you think that the authors may especially need support in writing/editing for English language. (If I can understand the methods and conclusions sufficiently, I prefer to review an article that appears to be from someone with less English proficiency based on the merit of the science, and to alert the Editor to a need for English editing).

Dear Authors,

Reminders:
• Do not say whether you recommended to accept, revise, or reject. That is the editor’s call.
• Use a friendly and professional tone, as if you were communicating to a co-author.
• Phrase recommendations in the positive in terms of what to DO (not what not to do).
• If you want to provide a suggestion but don’t consider it essential, consider the word “perhaps”, or consider not mentioning this altogether.
• You may want to point out something that is a flaw but that the authors cannot reasonably correct (e.g. adding more subjects for a study that is long past). In this case, your suggestions should be about how the authors should address this limitation and how it may influence their conclusions.
• Your recommendations are a “soft contract” that, if mostly implemented, are usually considered sufficient to then accept the manuscript, and so make sure that they are clear and defensible.

Thank you for your study of (summarize the question). Your manuscript (brief summary of methods), concluding that (summarize conclusions as you see them). This is (summarize your conclusions of the degree of contribution given prior literature). Strengths of your approach include (give a few strengths even if minor). (Possibly a few more sentences that provide perspective on intersecting issues, or which are the most important of the items below).

Major comments:

1. SHORT PHRASE NAMING THIS ISSUE. (List the most important improvement needed first. Explain why it is important, how it might alter the reading or conclusions, and how the author might address it.)
2. (Continue as above with items that you think really influence the correctness or impact of the manuscript.)

Minor comments:

1. (Here you will want to give recommendations that you think truly would improve the manuscript, but that are more along the lines of clarity, methodologic details that are warranted but wouldn’t be expected to greatly change the “story”, etc.).
2. Continue with comments as above.