How many manuscripts should you be reviewing?

/

Summary: Reviewing manuscripts is an important part of the scientific ecosystem and something we should all do. However, reviewing may not actually help our career all that much, and it's hard to know how much reviewing we should do relative to other activities. Here I suggest that early career scientists in particular should be careful not to review too much.

I think I probably review too many papers. I've been thinking about this issue in the context of my own priorities, and in terms of advice to give junior colleagues and trainees. How much should we be reviewing?

I did some informal (and obviously, unscientific) Twitter polls the other week. Here are a couple of results:

  • For PIs (n=81), 48% performed 6 or fewer manuscript reviews in the last 12 months. 31% performed 12ish, 10% 18ish, 11% 24 or more. (data)

  • For postdocs (n=95), 71% performed 6 or fewer manuscript reviews in the last 12 months. 18% 12ish, 6% 18ish, 5% 24 or more. (data)

When I asked what a good number of manuscript reviews for a postdoc to perform would be (n=155), 67% said 6/year or fewer, 28% 12/year. (data)

I myself am probably in the 20ish range, not counting revisions: according to my Publons profile I've done 33 reviews in the last 12 months, but that counts revisions (but not all of those papers have revisions).

Why do we review papers?

As a first step it might be worth thinking about why we review papers:

  • To be a good citizen
  • To read an interesting paper (either to stay up on the field, or have an idea what "the competition" is doing)
  • To support a colleague
  • To advance our career
  • To influence science in our field (thanks to @drugmonkeyblog for noticing I forgot this!)

These are all fine reasons, but it's worth being aware of them. And in particular, it's good to think about whether these goals are actually being accomplished (particularly career advancement).

What is a good amount to review?

Let me first list some ways people decide how many papers to review. I have not used any of these (successfully):

  • Say yes to everything. Bring invited is an honor, helps the field, and something you can put on your CV. Or maybe you are a masochist.
  • Keep the system in balance. For every paper you submit, an editor needs to get 2-3 reviewers. Therefore you should review 2-3 papers for every paper you submit to contribute your fair share.
  • Have an annual limit. Reviews take time and you decide to do no more than X per year. Keep a list and force yourself to say no to every single request once you've done X reviews. This will make you pickier so that you can say yes to "good" ones.

What I have typically done for the past few years is the following:

  • By happenstance have a brief respite from reviews
  • Feel guilty and agree to a random one on a topic vaguely related to my research
  • Immediately receive three requests for papers that are extremely interesting or from editors I know well or from a journal I'd like to support and/or be affiliated with
  • Say yes to all three new requests
  • Cry on the inside

Why we may not want to review quite so much

One obvious reason we may want to limit our reviewing is that it takes a fair bit of time to perform a useful review. I don't think I ever take less than 2 hours (and as a postdoc it frequently took me about a day). That's two hours less I have to spend on everything else I'd like to do.

Another important consideration is that reviews typically count for very little in career advancement. There may be exceptions, but although generally folks are expected to review "some" manuscripts to be a good citizen, I doubt the number of reviews performed come up at tenure and promotion committee meetings. This relates to the time commitment because it suggests that despite the significant time investment there is little concrete benefit.

[One possible way to change this situation is to increase the recognition given to reviewers, which may increase the benefit. I have a Publons account and think it's cool, but I don't think it's reached the point of actually affecting anything in my career.]

Given the current reality that reviewing does little for career advancement, another serious consideration is that women are more likely to participate in activities seen as service oriented than are men. If reviewing is not as highly valued as other activities (for example, publications or grants), then agreeing to review can put women at a disadvantage.

So, on balance, I would advise you early career scientists (postdocs and early assistant profs) to protect your time and review less. When you are comfortably established you can give back to your heart's content. For now, be selective in your reviewing and focus on activities that will advance your career.

The moral of the story

Of course, peer review is a good thing and we shouldn't take advantage of the system. I don't mean to suggest there are no benefits to reviewing—of course there are (and sometimes we do things to be a good citizen without expecting direct benefit, as we should). My worry is that the folks at the high end of the poll distribution who are doing 24 reviews/year are spending more time than they realize for less benefit, which is especially concerning for early career researchers.

Overall, I like the idea that we should review 3 papers for each first/corresponding author paper, because that way we aren't leeching off of the system. This rule of thumb also typically means that more junior folks will do fewer reviews, which is fair. If you are an especially productive early career researcher than you might consider writing an IOU to the universe and catch up on your reviewing after you get settled in a job.

(Obviously there are plenty of reasons to break this rule. But at least try to start with a rule! Or at least think about the broader picture before you accept another review.)

Now if you'll excuse me I have to go work on a review (true story).