[This post is an abridged version of the editorial in the November 2011 issue — the full text can be accessed here, available for free to all registered users. We welcome feedback on our editorials in the comments section below.]

What makes the ideal referee report?

There is no simple answer to this question. An author probably hopes for a quick report that is positive, or at least constructively critical. An editor will most appreciate a report that provides insightful comments and helps to inform a decision. The perfect report for a reviewer is more difficult to define; perhaps one that is not misinterpreted, and ultimately improves a manuscript.

Reports simply stating that a manuscript should be accepted or rejected, without providing any justification are rarely useful. It is, for example, unlikely that two brief reports that say a manuscript is ‘great’ and recommend publication without any compelling reasons to back up these statements will outweigh a thoughtful and well-supported report that highlights lots of technical flaws.

Based on our experience of the process — taken together with feedback from our authors and referees — we suggest the following guidelines that try to satisfy the needs of everyone involved.

Reports should begin with a short summary of the work in question. This serves to focus the review, clearly stating how the work is viewed by the referee and highlighting differences between how the authors and readers will interpret the work.

Reviewers should state upfront if there are parts of a paper that they are uncomfortable evaluating. They are, however, welcome to provide opinion on areas outside their own expertise as such information can be valuable in judging general appeal of the work. Authors should keep in mind that an individual reviewer may have been chosen to represent a particular point of view.

The summary should be followed with a discussion of what has gone before, in an attempt to define the advance that has been reported. These comments are most valuable when backed by references. Following this, a summary of both the merits and problems of the research is useful. This can be far more instructive than writing a report with a particular outcome (accept/reject/revise) in mind.

A good report should clearly distinguish between the claims made and their importance versus the evidence presented in support of those claims. This brings us to another issue frequently raised in criticisms of peer review: the need for additional work. Although requests for additional experiments are regarded by some as a ‘tyranny’, such requests are often quite reasonable.

With this in mind, it may be useful to divide suggestions into different groups. First, and most important, is work that is considered necessary to support the specific claims of the paper: omitted control experiments or requests for complete characterization. Second are those that may allow broader conclusions and improve the appeal to a general audience. Third are those experiments that are not essential, but might provide interesting avenues for future studies.

Proofreading is not the role of referees. However, the writing should be as clear as possible, and reviewers are encouraged to point out areas where language is too specialized and could be improved without detriment to the scientific content of an article.

We realize that reviewing places a heavy burden on a researcher’s time, so we are extremely grateful to the reviewers without whom Nature Chemistry could not function.

You can read full the editorial here (registration is free).

Steve

Stephen Davey (Associate Editor, Nature Chemistry)