Should you review?

Remember that there is always a cost in terms of your time – to anything that you do! The more you advance in your career (and the better you do), the more opportunities you will have. So begin to practice the art of evaluating whether any given activity is worth it.

If your advisor has asked you to assist with a review, chances are that your advisor has determined that this is the right time in your professional development and that the subject matter is a good fit. In other words, your advisor has already done the “worth it” calculation for you. Additionally, your advisor will assist you. Go for it!

What if the journal contacted you directly? Congratulations! This will most likely happen if you have a publication on a similar topic, and guess what – this means you are an expert! Truly – expertise always comes in a niche area and always in increments. That is, you will never know everything, even in a given area. Likely you are now minimizing your expertise to yourself. Don’t! Practice your confidence and wearing your new “expert” hat. It will feel a bit uncomfortable at first but feels better and better as you wear it more.

How do you determine the “worth it”? The gain will be in putting this on your CV and in the insight you will gain in terms of what makes an effective journal article. You will also likely learn something about the subject matter and/or ongoing research in your field. These are all pretty big benefits. The downside is in your time, and making sure that you can do an adequate job. Here are some things to consider –

• Is the journal indexed where it should be? For public health, check PubMed. If the journal has a lot of listed publications, you are a go. If the journal is not indexed, I recommend not reviewing. It likely indicates that you have been contacted by a non-reputable journal.
• What is the impact factor (an indication of journal quality) like? (You should know this term and how to find this info. https://en.wikipedia.org/wiki/Impact_factor. Finding impact factors can be super-easy for some journals: search on google and it will be displayed. But a thorough search for official published impact factors requires having paywall access to “Journal Citation Reports”, such as through the Web of Science. Higher impact factors are better. I’m not sure how high they range, but as I look Environmental Health Perspectives is about 9 and JAMA is about 47). Lower impact factors may mean that it is a new journal or a journal whose articles aren’t cited as much. It’s OK to review for a journal with a low impact factor at this stage of your career, but it is good to at least know one (imperfect) metric of the journal quality.
• Is the topic of continued interest to you? If you have any interest in the topic, review. If the topic reflects an old interest of yours that is truly distinct from your current interests, you may want to skip this, unless you are really wanting to beef up your CV.
• Do you have the time? A first review will take some time, both in reading, thinking, and writing, and in waiting for assistance. The time could be about that for a class paper, but of course will vary. The journal will often ask that you do the review in 2 weeks. It may be possible to ask for an extension (such as an additional 2 weeks), and it is OK to ask. As always, don’t do this if it will push you over the stress edge or if it will delay other work in a way that will then greatly disappoint you later.
• Have you completed any article critiques (such as in a classroom setting?) Article critiques are a good foundation to doing peer review and if you have not done any, you will need substantial support from someone (e.g. your advisor) in doing this review. If you can not obtain that support, don’t review.
• Are you able to do a quality review? It is ultimately up to you to assess whether you can go it alone or whether you need support. Consider the role (see below) and that it is not fair to the authors to review when your ability to do so is marginal. Talk to your advisor about whether you should go it alone. Here are some guidelines: If you are a post-doc or faculty member, go it alone. If you are a master’s student or doctoral student doing coursework, get support. If you are a doctoral student in dissertation phase and have published, you may be able to go it alone or may want support. If you need the support and your advisor cannot do this, and cannot recommend someone else who is available to do this, say no.

A Cheeky Confidential on Confidentiality and Conflicts

You have likely already been given access to confidential writing – the abstract sent in the invitation to review. And if you review you will see work in progress, maybe even with the full names of all authors. Later you may see the editor letter and reviews of others if you are included in the communication to the authors following the editor decision. So much good stuff to learn from!

So – will some of this influence your work? Likely, because human nature is to constantly learn and integrate all that we learn. This is unavoidable and honestly a perk of reviewing. But will you tell a colleague about this manuscript under review at any point until it is published? No. Will you save the pdf and use some of the sentences in your own work? Of course not! Will you contact the authors and/or tell others that you had a role in reviewing? Nope.

What about conflicts of interest? You may be the right person to review even if you know the author or have similar projects underway, because you are likely highly qualified in the subject matter and methods. But of course be on the lookout for whether any emotions may be there – feelings of competition, a personal grudge, or even the desire to help out a friend. If you have any hesitation about whether these feelings cannot be kept in check, it’s best to decline to review. (Reminder: this could be a career-influencing decision you help to make and ultimately the science and the public are best served by impartiality.)

You must use your strong inner compass about respecting others to sort out the nuances of confidentiality and conflicts. Seek advice from your advisor or others with experience in these areas whenever you are unsure.

Your Role as a Reviewer

Your role is ultimately to further the published science to better the public’s health and the world!

It is your job (together with the editor and other reviewers) to prevent the publication of junk. Junk = science that is flawed and/or conclusions that are not supported. Keep in mind that once something is published, it will obtain the sheen of truth, and will be cited, included in meta-analyses, and could be used to intervene upon, set policy, or otherwise produce real-world consequences.

Another role of a reviewer is akin to an unacknowledged co-author. You will be providing constructive feedback to improve the work and the clarity of the writing. Your comments may even have a mentoring role in helping the authors be better researchers and/or writers.

Choices: Accept, Revise, Reject

You will give a recommendation to the editor on whether an article should be accepted as is, considered again after revisions (some journals ask whether you think the requested revisions are major or minor), or whether to reject.

Accept = Accepted as is.
Rare. Very rare.

R&R = Revise and Resubmit. Considered again following revisions.
Common. Very common.

Remember that the editor and journal have already pre-screened the manuscript to assure that it is appropriate for the journal and deserves the burden they will put upon their reviewers. They may make mistakes, but they are not sending out everything that comes their way.

Also – because it is very likely that you will recommend a R&R, you will want to provide detailed comments to the authors on where and how to improve. Your specific recommendations will be taken seriously by the editor and the authors. In some ways your recommendations will form a “soft contract”, where it is understood that if the authors make the requested revisions and/or provide sufficient justification for their approach, it is likely that their manuscript will be accepted and published.

Reject
Use this category when the question that can be answered with the data and methods is not a sufficient contribution, when there is a fatal flaw, or when the writing is such a mess that you cannot make heads or tails of what was done.

What is a fatal flaw?
A fatal flaw is something that prevents the manuscript from being a contribution at all. In other words, the body of literature will be more correct (closer to truth) without this publication at all. Fatal flaws can be in the study design, measurement, statistical approach, logic of the conclusions, or in any aspect of the manuscript, but they are always so large that “the answer” provided is most likely wrong. Remember, when evaluating the degree of flaws and biases, there is a continuum of types of errors from minor to major and that no study is perfect. But also remember that once something is published that it has the sheen of truth. Glossing up a wrong result with the polish of publication can lead to real-world hardship. You have signed up to prevent this from happening to the best of your ability.

Who are you writing for?

1. The editor. Your primary intended reader is the editor – the person given the power to decide whether this journal will publish this article. The editor is looking to you as an expert who will give the time to advise on:
• Whether this question is important (a worthy contribution given the previous existing literature)
• Whether the question is answered with the data and methods
• Whether the study design and methods are appropriate (without major flaws or bias. Remember that no study is perfect).
• Whether the conclusions are appropriate to the question and results

Yet note – even though your primary reader is the editor and your primary role is to help advise on whether to publish this article as-is or following revisions –
You will not tell the authors whether you recommend an accept, revision, or reject, because it is ultimately up to the editor to make this decision based on your advice and that of the other reviewers.

And most of your writing will be penned to the authors, because this is where the detail is needed, and the editor is able to read your notes to the authors.

2. The authors. Your second intended readers(s) are the author(s). You will be informing the author(s) about how you view the contribution of the manuscript – the question and methodological strengths, and of course importantly, how it could be improved.

Brief Outline of a Review

Dear Editor,

Comments to the editor should be brief (1 paragraph) to provide punchy overview. The authors will not see this part and so you may also use this section to speak candidly (but professionally of course).

Thank you for the opportunity to review. This manuscript is a contribution (or not) because (explain). Major strengths are (summarize – can be the same summary used for authors). (Next state briefly why you made the recommendation you did, including perhaps the 1 or 2 biggest recommendations to improve and why).

(Also include here anything you don’t want the authors to see. This could be areas where you did not feel you could sufficiently review. Note that you are not expected to be expert on every aspect of the approach. You may also want to comment on the quality of the writing, and/or if you think that the authors may especially need support in writing/editing for English language. (If I can understand the methods and conclusions sufficiently, I prefer to review an article that appears to be from someone with less English proficiency based on the merit of the science, and to alert the Editor to a need for English editing).

Dear Authors,

Reminders:
• Do not say whether you recommended to accept, revise, or reject. That is the editor’s call.
• Use a friendly and professional tone, as if you were communicating to a co-author.
• Phrase recommendations in the positive in terms of what to DO (not what not to do).
• If you want to provide a suggestion but don’t consider it essential, consider the word “perhaps”, or consider not mentioning this altogether.
• You may want to point out something that is a flaw but that the authors cannot reasonably correct (e.g. adding more subjects for a study that is long past). In this case, your suggestions should be about how the authors should address this limitation and how it may influence their conclusions.
• Your recommendations are a “soft contract” that, if mostly implemented, are usually considered sufficient to then accept the manuscript, and so make sure that they are clear and defensible.

Thank you for your study of (summarize the question). Your manuscript (brief summary of methods), concluding that (summarize conclusions as you see them). This is (summarize your conclusions of the degree of contribution given prior literature). Strengths of your approach include (give a few strengths even if minor). (Possibly a few more sentences that provide perspective on intersecting issues, or which are the most important of the items below).

Major comments:

1. SHORT PHRASE NAMING THIS ISSUE. (List the most important improvement needed first. Explain why it is important, how it might alter the reading or conclusions, and how the author might address it.)
2. (Continue as above with items that you think really influence the correctness or impact of the manuscript.)

Minor comments:

1. (Here you will want to give recommendations that you think truly would improve the manuscript, but that are more along the lines of clarity, methodologic details that are warranted but wouldn’t be expected to greatly change the “story”, etc.).
2. Continue with comments as above.

I remember that “a-ha” moment when I learned that Beethoven (or Mozart) was not just sitting down with nothing and creating a masterpiece, but was instead working within a structure that dictated an outline or form of a certain type of music. The same is true of the major products of academic work – grants and publications. Even the process of how to formulate a next research question follows a regular process that can be taught, practiced, and understood. Doctoral training is in large part learning these forms and doing a big practice on the form (the dissertation). This is good news, because the form will support you and act as a platform and structure for you to build upon. Look for the forms. Ask about the forms. And most importantly, follow the forms. By mastering the form you will do your work with more ease, will help others know about your work, and ultimately, will be able to embellish or even break free of the form to achieve your goals.

Year 1. Focus on adjustment, learning the culture of doctoral studies, relationships, and courses.

In year 1, you are going to be transitioning to the culture of graduate school (and perhaps a new geographic area). You will likely be taking a lot of classes. Allow yourself the time to settle into these new routines. Be open. Get yourself grounded and happy in all areas of your life, as you will be spending several years in this endeavor. Do well in your classes and notice what you find particularly energizing (whether it’s a topic or a method). Be open and curious to the new humans around you and allow time for building relationships with them. Your classmates will be your future collaborators. Your advisor will be your guide. Your classroom instructors may be your future dissertation chair or on your dissertation committee. The staff member who heads up the graduate program will at minimum help you with the required paperwork and might help you with a real financial or personal dilemma. These persons are your companions along the way, and so you might as well enjoy human connection with them. Some may become lifelong friends.

Begin to pay attention to the following ways to get inspired and find dissertation topics. This is a long list, so the keyword here is BEGIN:
• Start an electronic or paper notebook solely dedicated to dissertation ideas and inspiration. Jot things down in this notebook regularly.
• Transition your resume to a CV. Look for models online and find one you like to follow. Keep it updated. Ask others for feedback on it, such as your advisor or even fellow students. You university might have an office to help with this, but make sure that they know you are developing an academic CV, not a resume.
• Establish an identity on academic/professional social network sites such as LinkedIn or ResearchGate. Begin to connect with others.
• Identify 2-3 top journals in your field and set up so that their table of contents are delivered to you (email). Regularly browse the table of contents.
• Read your advisor’s publications from the last 4 years. Ask yourself what is important about them and how they were practically accomplished. If you aren’t sure, ask your advisor.
• Attend a journal club that interests you or START a journal club that interests you. A journal club meets regularly to read and discuss publications of interest. At the beginning you might seek to simply understand the publications deeply.
• Identify local and national/international conferences of interest. Attend those that are low cost/low time as soon as you can.
• Attend EVERY in person seminar that is remotely related to your field. Seminars are useful because you will be exposed to topics before published, and moreso than publications, seminars often portray the big picture trajectory of a field and/or tell a research “story”. Always get something out of a seminar. If you are bored with a topic, ask yourself why. If the speaker is not great, take actual notes on why and what could be better. If the speaker is wonderful, again, take notes on what made it so. If the topic really interests you, both record this and also ask yourself what it is about the topic that energizes you.
• Identify webinars that you can attend (often for free). These are especially important if there are few in-person seminar options in your field.
• Find ways to keep up on how the news media may be covering your topic. This might be a news aggregator delivered to your email.
• See if there are any “lay” books that address your interest. This can be a great way to get the “big picture” of an area and also get inspired on what is of interest.
• About once a semester, tell your advisor about your current interests and ask for direction in terms of what to read/who to talk to.
• Set up “informational interviews” with experts in your field. Do not shy away from doing this. If this is difficult for you, find out why and address this. Ask your advisor who to start with. Be prepared by reading some of their publications ahead of time. Google “informational interview” to be more prepared. Don’t hesitate to ask them what inspires them about their area and what are key gaps in their area that need to be filled. These gaps might be dissertation ideas.

Year 2. Focus on courses, writing, and finding inspiring ideas.

Further your skill sets and deepen your knowledge, whether through classes, lab rotations, or your Research Assistantship. Do well in your classes. Take a class ideally on grant writing, or if this is not available, on academic writing or any sort of writing. The grant writing class should not only cover writing skills, but the foundations of how to select an answerable and worthwhile research question. This skill set is key and will help you develop your dissertation. In addition to spending MORE time on the list for Year 1, add:
• Assist in a manuscript review for a journal. Ask your advisor for opportunities to do this.
• Read a grant proposal in your field. You will need to work with your advisor to get access to one or two. Read actively to figure out why the scientific question was worthwhile, what practical decisions were made in designing the work, and what “size” the question is (and whether it seems large or small in scope to you.) Keep in mind that most students envision a dissertation project that is way larger than is feasible.
• Explore funded grants in your area of interest. One great way to find these is the NIH Reporter. Note that grants show you the research that is ahead of what is currently published.
• Read a Strategic Plan for a funding agency in your field, for example, an NIH institute like the National Institute for Environmental Health Sciences. Experts at these agencies are thoughtful about knowing work that is published and funded, and what should come next to make the world a better place.

In a nutshell, doctoral programs train you to be a contributing citizen of one or more academic disciplines. This is huge. It means that (if the program is successful), you will have the foundational knowledge and skill sets, and will understand the culture and practices to contribute knowledge to the world. You will have demonstrated your first contribution of new knowledge to the world – this is your dissertation. You will understand the importance of participating within a community of scientists – a “field”, by being able to identify and digest existing work so that you can leverage your contribution on top of what already exists. This will launch you and your new work several steps upward, so that you aren’t recreating the wheel, and can therefore make a bigger contribution. You will learn how to think practically – how to ask a question that can be accomplished within time and money constraints, which are ever-present. You will learn the strategy of how to communicate why your work is important and why it should be funded or shared with the world through publication. You will begin to participate in making decisions about whether others’ work is funded or shared with the world. You will also have an enhanced ability to continue developing after the doctoral program is over. You will BE a scientist. You will HAVE expertise. You will KNOW how to move forward.

“A Better Place” Blog consists of my words to support academic trainees – especially doctoral students, in all disciplines. Here you will find content ranging from practical tips, writing advice, tools of the trade, and philosophies about being in graduate school (and beyond) and also being happy! Becoming a contributing citizen of the world of knowledge is a fabulous way to grow your potential and make the world a better place. You can DO this!

On December 1, 2019, I renamed this blog “A Better Place”.

I believe that researchers are motivated by, and obligated to, make the world a better place. Along the way, let’s make your university a better place to work and grow. Maybe even let’s allow the space you really spend time – your mind, to be a better place.

(I post at irregular intervals, when I notice something in my own work or working with students and believe that others could benefit from guidance. Subscribe button below right).

(The fine print: While much of this blog will speak to many disciplines, there are differences in culture and practices. I write grounded in public health.)