Around the time Daniel Ucko started work as a full-time editor at Physical Review Letters in 2004, the US government ordered his employer – the American Physical Society (APS) – not to peer review and publish articles submitted from countries, such as Iran, that were facing economic sanctions. (The APS and other scientific organizations refused.) Ucko found it interesting that, lumping peer review with trade, the US was explicitly recognizing that the process generates value.

Ucko also found it curious that arXiv, then over a decade old, wasn’t supplanting peer-reviewed journals. Nor has it since. Peer review, he decided, is a process that produces value. “I began to wonder,” Ucko told me recently, “about the components of that productivity.”

Keen to explore the conceptual basis of peer review, in 2011 Ucko also became a full-time graduate student in the philosophy department at Stony Brook University, where I am a professor. Ucko found that a philosophical training helped him understand peer review and that his editorial and scientific experiences – he got a PhD from University College London and did a postdoc at the University of Birmingham in the UK – gave him new perspectives on traditional philosophical issues.

Take anonymity.

Don’t you know who I am?

Anonymity has a bad rap among philosophers, who think it stimulates inauthenticity and dishonesty. In his 1846 treatise The Present Age, the Danish philosopher Søren Kierkegaard attacked the press of his day for publishing anonymous articles. Anonymity, Kierkegaard argued, allows people not to commit themselves to their statements and encourages thoughtless opinions, rash judgment, innuendo and slander. More recently, the Berkeley philosopher Hubert Dreyfus has applied Kierkegaard’s arguments to the Internet, pointing out, for instance, anonymity’s role in trolling.

Peer review somehow inverts this effect, Ucko realized, making anonymity productive, not corrosive. Ucko began to see the reasons why thanks to a famous thought experiment by the American philosopher John Rawls called the “original position”. To design a political system with maximum fairness, Rawls argued, you have to ask people to do so behind a “veil of ignorance” – not knowing what particular characteristics (such as gender, race, class, or level of education) the designers themselves would have in it. That requirement maximizes impartiality by forcing social planners not to structure the system to favour their particular circumstances.

“I thought that was an analogue to peer review,” says Ucko, who went on to explain his thinking in a paper entitled “There is no ‘I’ in referee: why referees should be anonymous” at the 2015 March meeting of the APS. “If you create a situation where it is the author who is behind the veil and doesn’t know who the referees are or their perspectives, you are encouraged to write a paper – laying out your method and procedures in detail to no-one in particular – that is maximally compelling.”

Another philosophical concept important to peer review is tackled in Lorraine Daston and Peter Galison’s book Objectivity. The authors argue that the scientific quest for objectivity is driven largely by a distrust of ourselves – by our own subjectivity. But the objectivity aimed at in the peer-review process, Ucko thinks, is of a special kind. It does not mean trying to devise a procedure that somehow allows you to get behind the experimental process to grasp what’s happening. It’s rather the opposite: to articulate the experimental process in as public a manner as possible to optimally tap the expertise of others.

Balancing interests

The late physicist John Ziman, of the University of Bristol, once called peer review “a highly reflexive and convoluted social activity”, involving a balance of interests between three constituencies: authors, editors and referees. The balance is achieved, Ziman wrote, because scientists have to play each role. “It is,” he declared, “as if every citizen must sometimes be the accused, sometimes the judge and sometimes the jury in a succession of criminal trials.”

“It’s more complicated,” says Ucko, “for the pillars are not fully interchangeable.” Ziman’s image, for instance, doesn’t have room for professional editors, such as Ucko himself, who don’t do research.

Still, Ziman’s remark helps explain how anonymity can function productively. The anonymity in Kierkegaard’s press example is one pole of a pair: reader and anonymous writer. Ziman’s image makes clear that it’s a three-way relationship in a scientific publication, where the referee is anonymous to the author but not to the editor, who is able to interpret the referee’s remarks. It also explains why scientists should prefer learned-society publishers over for-profit publishers, whose editors are less beholden to the scientific community.

Ucko has organized a session, chaired by me, at next month’s APS meeting to focus on the role of trust in peer review. That trust is lodged not necessarily in a particular editor or even journal but in the system, and is generated by knowledge of how the system operates. An author knows, for instance, that it’s in the best interests of editors to replace an irresponsible referee. Other speakers include Harvard University science historians Melinda Baldwin and Alex Csiszar as well as Jamie Hutchins, a director at IOP Publishing, which publishes Physics World.

The critical point

Richard Horton, editor-in-chief of the Lancet, once wrote that it is a “mistake” to view peer review as anything more than “just a crude means of discovering the acceptability – not the validity – of a new finding”. We tell the public, Horton continued, that peer review is “a quasi-sacred process” that makes science objective, when we know that it’s in fact “biased, unjust, unaccountable, incomplete, easily fixed, often insulting, usually ignorant, occasionally foolish and frequently wrong”.

But, Ucko argues, we also know that peer review is productive. If we begin by examining that productivity, we’ll understand the process better. “The institution of peer review,” he says, “is a fascinating philosophical laboratory.”