Collections of essays often have an uncertain status. In the absence of any overarching message, they stand or fall on the quality of the writing and the author’s ability to offer regular jolts of insight and entertainment. When (full disclosure) David Kaiser told me last summer that he was putting the finishing touches to his anthology – Quantum Legacies: Dispatches from an Uncertain World – mostly reworked from pieces published already, I had little doubt that it would meet that challenge. 

Both a physicist and a historian of science, Kaiser has shown himself for years now to be an astute chronicler of the subject: able to explain arcane phenomena with a deft touch, as well as to weave the deeper context around the discoveries and developments of the discipline. What sets him apart from most popularizers is that he does not simply head for the flashiest topics but writes eloquently about the real-world business that engages most physicists: funding streams and industrial support, intellectual fashions and reputations, teaching and publishing. 

Quantum Legacies does not disappoint. As the title suggests, many of these chapters concern aspects of quantum theory, from the contributions of the mercurial Paul Dirac to Kaiser’s own research, alongside Anton Zeilinger, to test John Bell’s ideas about the nature of quantum entanglement, using individual photons of starlight captured in a telescope on La Palma in the Canary Islands. Along the way, we hear about the emergence of the search for extraterrestrial intelligence (SETI), the saga of the US gravitational-wave observatory LIGO, and why creationism hasn’t plagued cosmology as much as it has evolutionary theory.

One of the standout chapters might sound unpromising on paper: it concerns the changing nature of physics textbooks, specifically those teaching quantum mechanics. But within it is a metaphor for the entire discipline. In the 1920s and 1930s – while theoretical physics as a formal subject was still relatively young and quantum mechanics was nascent – there was a sense that students were learning a kind of artisanal craft, passed down through mentorship as “a great adventure in human understanding”, as J Robert Oppenheimer wistfully recollected. Quantum mechanics was then considered to demand engagement with philosophical, as much as mathematical, problems. In the 1930s, students at Caltech were expected to answer questions about quantum foundations that are still unresolved, and are a cause of furious arguments today: “What is [the] interpretation of ψ(x)? Discuss the nature of observation in quantum mechanics.”

But by the 1950s it was a different story. The textbooks had no time for such frippery; you had just to learn the techniques and apply them – famously, to shut up and calculate. “Where once-fabled teachers like Oppenheimer had relished talking through thorny conceptual challenges with small groups of students,” Kaiser writes, “instructors after the war – their intimate classrooms by then replaced by large lecture halls… – increasingly aimed to train quantum mechanics: skilled calculators of the atomic domain.” Richard Feynman’s famous books of lectures, while rightly revered for their clarity and panache, exemplify that no-nonsense attitude. Interpretational issues, Feynman wrote in the lecture notes to his graduate course on quantum mechanics, were “in the nature of philosophical questions” – and by implication, not important, or at least “not necessary for the further development of physics”.

And this utilitarianism, as Kaiser points out, echoed the almost factory-like over-production of physics graduates: a bubble inflated by fears of Soviet competition, and which burst around 1970 when funding plummeted. Stimulated by Science, the Endless Frontier (1945), Vannevar Bush’s famous manifesto for the value of basic science in supporting economic growth and national security, this post-war drive to train physicists who could “get the job done” relied largely on defence funding. It was fuelled by the notion that the Second World War had been “the physicists’ war” – a phrase coined for other reasons, but which became attached to the idea that victory had hinged on the Manhattan Project.

It is a breath of fresh air to see physics writing like this: lucid and friendly, sober and thoughtful

As far as quantum mechanics was concerned, Kaiser’s splendid 2011 book How the Hippies Saved Physics (Physics World Book of the Year 2012) recounts how a small group of counter-culture rebels in California during the 1970s revitalized interest in the philosophical foundations, revisiting the arguments of Bohr, Einstein, Schrödinger and Heisenberg over what the remarkably effective formalism actually said about reality. All the same, for a young researcher to take an interest in such questions was frowned upon until only the past decade or two. Now quantum information technologies are showing that they were not after all irrelevant to the practical concerns of engineers.

Kaiser is also good at teasing out how such sociological considerations have influenced the questions physicists ask and the reception of the answers they give. In one chapter he explains how the gulf between particle physics and cosmology in the 1970s hindered an appreciation of the link between the alternative gravitational theory of Robert Dicke and Carl Brans, and the Higgs field proposed by Peter Higgs – an idea that now suggests a role for the latter in cosmic inflation. “Contours of intellectual life can be reshaped by rapid changes in institutions and infrastructure,” says Kaiser, “ultimately shifting the boundaries of what young physicists come to find compelling or worth pursuing.” That can never be stressed enough: however obvious the questions might seem, they will have been socially selected and moulded. 

It is a breath of fresh air to see physics writing like this: lucid and friendly, sober and thoughtful, and willing to trust the reader’s engagement and intelligence rather than demanding the former and underestimating the latter. It’s also the case – probably inevitable in such a collection – that Kaiser is not always writing for the same audience. The description of the symmetries of the Standard Model, for example, is superb popular science (“teeming collections of atoms, which are mostly empty space, their subatomic constituents acquiring heft from the symmetry-preserving whirl of a gluonic quantum dance”) – but speaks to a different demographic from the accounts of post-war physics training or the fate of the Superconducting Super Collider. The book is, too, not without a little of the overlap that often results from such patchwork assembly. Nonetheless, it is hard for me to imagine any physicist who wouldn’t enjoy the fine cloth from which it is cut, nor the pleasing effect it makes.

• 2020 University of Chicago Press 360pp £21.00hb