How far can people trust what scientists say about science? In principle, the question does not make sense. Science is about investigating the natural world, and we should not have to take anyone’s observations purely on trust. Nature is what it is, and trust ought not come into it. In practice, however, few scientists have the expertise or resources to reproduce their colleagues’ work, and this is certainly the case for the “educated members of the public” who make up the audience for popular-science books. How can these untrained but intelligent, sci-curious folk tell when they are reading about exciting advances, and when they are being fed rubbish?

The discouraging truth is that they cannot – at least, not always. Physics, astronomy and cosmology are far too weird for common sense to be a reliable guide and too many scientists have made spectacular mistakes for readers to trust their authority absolutely. But in lieu of a reliable “scientific nonsense detector”, anyone with an interest in the accuracy of popular-science writing should read Alan B Whiting’s Hindsight and Popular Astronomy, which offers an excellent guide to answering the question “How far should I believe what I am told?”.

Whiting is currently a visiting astronomer at the University of Birmingham in the UK, and as he explains in the book’s introduction, he might have addressed the problem by performing his own research to check the accuracy of recently published books. However, the most interesting questions in modern astronomy – from “What is dark matter?” to “Are there other universes?” – currently have no answers, so there is no guarantee that his conclusions would be better-founded than those of anyone else. So instead, Whiting decided to concentrate on older books, using hindsight to evaluate how well their explanations have stood the test of time.

Hindsight is not a perfect tool, but Whiting has a good grasp of its limitations. He is not interested, for example, in pointing and laughing when a 19th-century astronomer offers a wholly erroneous account of how the Sun shines. But while it would be unfair to criticize, say, Sir John Herschel for not guessing that the Sun is a gigantic nuclear furnace, Whiting argues that it is reasonable to object if scientists seem very sure of their incorrect explanations. Astronomers who are often wrong, but never in doubt, are doing a great disservice to their readers. Conversely, if readers are told that a certain statement must be taken with a very large grain of salt, he reasons, then they will not have been misled if later research shows that the statement was wrong.

The nine books that get Whiting’s hindsight treatment were published between 1833 and 1944. Whiting devotes a chapter to each book, with additional chapters in-between that describe the state of astronomy at the time of publication. Though his choice of books is by no means comprehensive, it is not random either. All of the books were “popular” in both senses of the word: they were well received in their day, often going through multiple editions, and they were written for the public rather than for students or other astronomers.

Another common factor is that all six authors (three of them appear in Hindsight a second time, for different books) were recognized scientific authorities, not journalists or amateurs. Herschel, George Airy, Arthur Eddington and James Jeans were all giants of English astronomy, and though the Canadian-American Simon Newcomb was largely self-taught, by the time he wrote Popular Astronomy in 1878 he was among the world’s pre-eminent mathematical astronomers. Even the least significant of the six, Robert Ball, was a Cambridge professor with a knighthood. Whatever the reasons for their errors, lack of intelligence is surely not among them.

But amid these similarities, Whiting finds some fascinating differences. One is his authors’ concept of who, exactly, constitutes “the public”. In the 1833 edition of Treatise on Astronomy, Herschel assumes that his readers will know what a versed sine is, and in the 1869 edition (which Whiting evaluates as a separate work) he blithely assures them that a certain result can be checked with a trivial half-hour’s calculation. The contrast between Herschel’s books and the equation-free zone of modern popular-science writing could hardly be more marked. Yet if this is evidence of “dumbing down”, its origins definitely predate the 21st century: Airy does not even bother with algebra in his Popular Astronomy lectures of 1848.

The most significant difference between the starry six, though, is how they measure up to Whiting’s demanding standards of accuracy. Briefly, Eddington and Newcomb do very well indeed, while Airy and Herschel escape with a light scolding. Jeans, however, is a disappointment, and Ball an unmitigated disaster whose “embarrassment” of a book “falls far short of even everyday standards of logic and consistency”. Among other errors, Whiting accuses Ball of egregious hand-waving, setting up “strawman” arguments, and repeatedly assuming that “we don’t know of any” is the same as “there aren’t any”. Ball’s book, In the High Heavens, was published in 1893, but sadly, Whiting’s criticisms could apply equally well to some modern authors.

Whiting dismisses Ball in a handful of pages, but Jeans’ mistakes are more interesting. Of the 1929 edition of Jeans’ The Universe Around Us, Whiting writes that “it is hard to know just how to approach such a mixture of penetrating insight and sheer error”. Notable faults include Jeans’ “proof by assumption” that stars are made of liquid and that their energy comes from annihilating protons and electrons. The annihilation reaction, in particular, had never been observed and there was no evidence for it, but Jeans assumes it to be true. He also makes errors even when contradictory evidence existed – to the extent that, in Whiting’s view, “it is hard to escape the conclusion that Jeans…was not only unfamiliar with much of the observational material available, but uninterested in anything beyond a superficial glance at features that might be used to support his theory”. Strong stuff.

In fairness, Whiting concedes that Jeans was writing at a time when the underlying physics of stars and other heavenly bodies was in flux, thanks to the ongoing quantum revolution. Many of the questions Jeans considers could not have been formulated by Newcomb, never mind Airy or Herschel. But there is a right way and a wrong way to write about science when all or part of your material is speculative, and Whiting deftly illustrates this by comparing Jeans with his Cambridge colleague Eddington. The third version of the latter’s Stars and Atoms was published in 1928, and according to Whiting, Eddington “proceeds by pushing known theory as far as it can go”, rather than charging wildly into the unknown. Eddington also hedges his speculations with appropriate caveats, and Whiting commends him for his “willingness to be baffled” and humility in admitting that there are some things he simply does not know. Though Eddington does get a few things wrong, there is enough doubt in his writing to alert readers when he is venturing onto shaky ground.

Whiting’s clear prose and bone-dry humour make his book a real pleasure to read, and its subject matter will surely fascinate science historians and lovers of popular-science books. Yet I would argue that Hindsight – which appears to have been published cheaply, and with essentially no publicity – deserves a much wider audience. The question of how much trust the public should place in scientists is not some arcane topic of purely historical interest, and it is certainly not limited to astronomy. The debate about climate change, for example, has been plagued by mistrust, with some self-proclaimed “sceptics” refusing to believe anything that climate scientists tell them. Medical science also suffers from a trust deficit, thanks to several well-publicized cases in which spectacular claims were contradicted by later research (“Red wine prevents cancer! Wait, no, it causes it!”). By setting out the do’s and don’ts of writing about uncertain science in the “safe” context of astronomical history, Whiting’s book does a great service to anyone who wishes to apply these lessons to more controversial fields.