Albert Einstein’s persistent opposition to quantum mechanics is a familiar, if still somewhat surprising, fact to all physicists. It was first voiced in 1926 in his famous comment written in a letter to Max Born that “Quantum mechanics is certainly imposing. But an inner voice tells me that it is not yet the real thing. The theory says a lot, but does not bring us any closer to the secrets of the ‘old one’. I, at any rate, am convinced that He is not playing dice.”

From then, until Einstein’s death in 1955 – as he struggled without success to find a unified theory of electromagnetism and gravitation – his opposition never wavered, and made him an increasingly isolated figure in physics. “To Einstein, probabilities were just a sign of gaps in our understanding,” David Bodanis concisely observes in his latest book – Einstein’s Greatest Mistake: the Life of a Flawed Genius – the title of which refers to this opposition.

Less established are the reasons, both intellectual and personal, for Einstein’s resistance. According to Bodanis, they lie in the history of the cosmological constant, unwillingly introduced by Einstein in 1917 into his 1915 field equations of general relativity. Added as a fudge factor with a repulsive effect to balance the attractive effect of matter, the cosmological constant was meant to produce a static solution for the universe: a concept that in 1917 seemed evidently correct to astronomers. When subsequent observations of galaxies by Edwin Hubble and Milton Humason proved that the universe is actually expanding, Einstein willingly abandoned the cosmological constant around 1931 and reverted to his original field equations. He even, apparently, referred to the cosmological constant as “the greatest blunder of my life” (a comment quoted by Bodanis without reference to its somewhat doubtful source). But as a result of his volte-face, says Bodanis, Einstein became increasingly convinced of the superiority of his intuition over experiment – a view that, by the 1930s, hardened into dogmatic opposition to quantum mechanics.

Telling support for this stance, oddly unmentioned by Bodanis, comes from an Einstein lecture, “On the method of theoretical physics”, delivered at the University of Oxford in 1933, not long before he emigrated to the US. Here Einstein controversially stressed the importance of mathematics over experiment in devising physical theories by saying that “Experience can of course guide us in our choice of serviceable mathematical concepts, [but] it cannot possibly be the source from which they are derived; experience of course remains the sole criterion of the serviceability of a mathematical construction for physics, but the truly creative principle resides in mathematics. In a certain sense, therefore, I hold it to be true that pure thought is competent to comprehend the real, as the ancients dreamed.”

Nobel laureate Steven Weinberg would appear to agree with Bodanis. In “Einstein’s search for unification”, an essay Weinberg contributed to my book, Einstein: a Hundred Years of Relativity, he concludes that because general relativity had been guided by an existing mathematical formalism – the Riemann theory of curved space – perhaps Einstein had acquired “too great a respect for the power of pure mathematics to inspire physical theory. The oracle of mathematics that had served Einstein so well when he was young betrayed him in his later years”.

The most original aspect of Bodanis’ book is its attempt to explain difficult concepts in ordinary language, without, of course, resorting to mathematics. For instance, Bodanis compares curved space in general relativity to two Finnish skaters who head for the North Pole, using compasses to carefully skate in parallel, but are inevitably “pulled” together until they crash into one other at the pole. He also pictures Heisenberg’s understanding of uncertainty at the subatomic level as the experience of an audience at a 1920s Berlin operetta. The audience can work out general patterns among the actors from the type of clothes they change into for each act, without knowing exactly what the actors are doing backstage. “Heisenberg would have been convinced that what had happened backstage was inherently a blur,” suggests Bodanis. Whereas from Einstein’s perspective, “each individual actor had to be changing his or her costume”.

Less original, though also engagingly integrated with the book’s physics, are its biographical elements. These cover not only Einstein but also others such as his second wife Elsa Löwenthal, his lifelong friend Michele Besso and his sparring partner Niels Bohr. His undergraduate physics teacher in Zürich, Heinrich Weber, who Einstein rightly regarded as well behind the scientific times, told him “You are a smart boy, Einstein, a very smart boy. But you have one great fault: you do not let yourself be told anything.” For much of Einstein’s life, this self-confidence was without question a vital strength, but in his later years, argues Bodanis, it became a handicap.

Yet, as the essentially respectful Bodanis admits, even Einstein’s opposition to quantum mechanics could be fruitful. His 1935 so-called EPR paper, “Can the quantum-mechanical description be considered complete?”, written with Boris Podolsky and Nathan Rosen (neither of whom is named by Bodanis), provoked a fellow-sceptic, Erwin Schrödinger, to come up with the technical term “entanglement” and his tantalizing “cat” paradox.

Schrödinger, unlike Einstein, eventually accepted quantum mechanics as a profoundly useful method of calculation. However, the debates about its correct physical interpretation launched by the great, if flawed, Einstein, are very far indeed from being conclusively resolved. “What is quantum theory, a century after its birth?” asks Carlo Rovelli in his recent book Reality Is Not What It Seems: the Journey to Quantum Gravity. “An extraordinary dive deep into the nature of reality? A blunder that works, by chance? Part of an incomplete puzzle? Or a clue to something profound regarding the structure of the world, which we have yet to fully decipher?”

• Enjoy the rest of the April 2017 issue of Physics World in our digital magazine or via the Physics World app for any iOS or Android smartphone or tablet. Membership of the Institute of Physics required