Music and physics might seem like polar opposites, one having great emotional potency and the other being a cerebral subject of equations, theories and deductions. Both, however, benefit from improvisers – people who stand on the shoulders of giants, taking earlier triumphs and building on them to create something new. For me, analogies like these, which draw parallels between physicists and jazz musicians, are the most fascinating revelations in Stephon Alexander’s book The Jazz of Physics – despite the book’s eye-catching promise to reveal “the secret link between music and the structure of the universe”.

The book begins autobiographically, with a description of Alexander’s early childhood in Trinidad and his formative years in New York’s Bronx neighbourhood. Writing in a fluent and easy-to-read style, Alexander describes how he turned down the opportunity to pursue fame and fortune through hip-hop and jazz in favour of his other passion: theoretical physics. Now a professor at Brown University in the US, he remains an amateur saxophonist, and has been known to take his latest theoretical musings to jazz gigs, to scribble down ideas between sets and so improvise new ideas in physics.

Similarly, my reading of The Jazz of Physics was accompanied by music mentioned in the book, such as My Favourite Things by John Coltrane and Ambient 1: Music for Airports by Brian Eno. For those unfamiliar with the latter’s work, suffice to say that Eno has been one of the world’s most influential electronic musicians and music producers since the 1970s; the most exciting message I received on Twitter all last year was a photo showing my popular-science book on a bookshelf in the toilet of Eno’s studio. Alexander, though, trumps my anecdote by actually taking us into that studio. He also discusses improvisation with great saxophonists, such as the controversial free jazz pioneer, Ornette Coleman.

A common misconception of scientific research is that it is all about people working alone to achieve dramatic “eureka!” moments. As the large cast of supporting characters in The Jazz of Physics shows, this is not how most of science progresses. To begin with, it seemed to me that Alexander was overly keen to namedrop well-known musicians and physicists, but then I realized that this is just a reflection of how he works. Whether Alexander is playing on his sax or doing physics, the role of the other musicians/scientists supporting his improvisation is vital.

Another misconception is that scientific research is a linear process with a simple story line. Throughout the book, Alexander demonstrates how important it is for physicists to play with ideas, to take things that might at first seem daft and see what they reveal. The book is very much a personal story, but it also illuminates some of the current arguments about the value of “blue skies” research versus work with more immediate impact. Jazz doesn’t move forward if every improvisation is just a pastiche of past maestros. It needs to build on the framework of past players to take new and unexpected avenues. The same is true of science.

There is plenty of science in this book as well, with Feynman diagrams, string theory and cosmology being just some of the areas covered. Sometimes, though, the autobiographical narrative means that topics are introduced and then not fully developed because Alexander’s career moved in a different direction. This is certainly a faithful portrayal of how most scientists work: just as a good jazz improviser draws on many different influences, Alexander’s science has drawn on a broad range of fields in theoretical physics. And it is fine for those who have some familiarity with the subjects, as I expect most readers of Physics World will. But at times, I found it a bit unsatisfactory, since it meant that some topics are covered quite briefly, and some important concepts aren’t fully explained. If a reader with little physics background were to approach this book, it would be like someone brought up on 1950s rock and roll going to hear an Ornette Coleman free jazz gig for the first time.

Putting this aside, though, the book’s attempt to bring together modern jazz and modern physics strikes me as admirable. The Jazz of Physics riffs on the idea of a rhythmic universe, where the cosmos is following a cyclical pattern of expansions and contractions interspersed with big bangs. After each big bang, a new universe is created that may or may not have the capability to develop life. Alexander’s musical analogy is the cycles of improvisation in Coltrane’s Giant Steps, where the start of each cycle gives Coltrane an opportunity to develop a new idea and form a new musical universe. It’s an intriguing comparison, and it certainly seems fresher than drawing analogies between classical music and classical physics, where there is a long history of scientists telling stories about Pythagoras and the “music of the spheres”. Time to put on some Coltrane and riff some new research ideas?

• Enjoy the rest of the May 2016 issue of Physics World in our digital magazine or via the Physics World app for any iOS or Android smartphone or tablet. Membership of the Institute of Physics required