Until the end of the 17th century, writes art historian Kenneth Clark, artists thought of light as “an act of love”, for it seemed to reveal, brighten and intensify nature. Light was the principle of epiphany, the self-disclosure of the world and its beauty.

Isaac Newton’s famous 1704 book Opticks seemed to massacre that picture. Subtitled A Treatise of the Reflections, Refractions, Inflections and Colours of Light, it saw Newton treat light as just a mechanical phenomenon governed by mechanical laws. Colour is subjective, a sensation produced after light rays strike the eye. Sunsets, rainbows and moonbeams are explained by geometry in action, followed by the brain in motion.

Many poets and artists reacted badly. Keats said Newton had “unwoven the rainbow”, while his fellow poet Thomas Campbell protested that science had replaced “lovely visions” with “cold material laws”. Few of Newton’s critics went as far as Johann Wolfgang von Goethe (1749–1832). Not content with complaining, he undertook experiments to write his own, artistically friendly, Farbenlehre (or Theory of Colours) in 1810.

Colour wars

Aside from his prolific poems, plays, novels and drawings, Goethe was an avid scientific researcher. He collected botanical, geological and zoological specimens. He owned electrical and optical instruments, and composed treatises on anatomy and botany. Goethe’s studies convinced him that the widely distributed variation of natural forms (plants, say) were related and flexible, as if they derived from a single unity or archetype (Urpflanze).

Animal forms also seemed to derive from an archetype. Yes, features such as horns, limbs and tails might vary, but only in a way that unifies them with each other and their surroundings. Goethe’s ideas about morphology – he coined the term in fact – crudely prefigure the discovery of evolution. The task of biological science, he thought, was to discern these archetypes, which make all biological forms intelligible and which have no further explanation, and to understand how they vary with the environment.

Goethe saw archetypes as reaching throughout nature. The title of his 1809 novel Elective Affinities, for instance, is derived from a scientific term (from the era before the periodic table) about the tendency of certain substances to relate with others. The novel transposes the idea to human relationships, exploring parallels between human and chemical interactions.

When Goethe published Farbenlehre the following year, he likened Newton’s theory of colours to an old fortress hastily erected with “youthful impetuosity” and subsequently supplemented with “towers, battlements and embrasures” to seem imposing. We honour it, make pilgrimages to it, teach it – but in the end it is “uninhabitable”. The only people who still live in it are “a few aged soldiers”. Goethe promised to “raze the Bastille”, and erect a new, modern fortress using a truly scientific procedure.

Newton’s old fortress is uninhabitable, Goethe continued, because colour is also an archetype – not a cranial byproduct but an “elemental natural phenomenon” that humans directly apprehend. It arises through the mixture of light and darkness in the presence of a “turbid” medium, such as air or moisture, and exhibits myriad variations depending on viewing conditions. Goethe’s book even included colour plates to be viewed through prisms to observe such properties as intensifications, neutralizations, refractions and halos of light.

Goethe’s Farbenlehre did not dent the Bastille. Newton’s Opticks had defects, including his division of the rainbow into seven colours and his denial of the possibility of achromatic telescopes, but these errors were rectified by later scientists. Yet Goethe’s book inspired many. Beethoven asked to read it. J M W Turner incorporated aspects into his work, as did Wassily Kandinsky and other abstract painters. Philosophers who seriously investigated its ideas included G W F Hegel, Arthur Schopenhauer and Ludwig Wittgenstein. It was a key text for the “anthroposophist” movement – a spiritualist philosophy founded by the esoteric philosopher Rudolf Steiner in the early 20th century.

More recently, speaking at a 1968 conference on quantum mechanics in Cambridge, UK, the German physicist Carl von Weizsäcker invoked Goethe’s ideas in trying to interpret concepts in quantum theory, while Werner Heisenberg sympathetically mentioned Goethe’s work on colour in 1971. But why did so many people find Goethe’s failed assault on Newton fascinating?

The critical point

Many physicists have been bitten by the colour bug – including Erwin Schrödinger and Richard Feynman, who devotes two chapters to it in Lectures on Physics. But to explore colour, Feynman admits, requires going “beyond physics in the usual sense”. Those inspired by Goethe felt the need to rescue not only the exploration of colour, but other science as well, from “physics in the usual sense”; from reductive explanations that replace experience with mechanics. No merely scientific explanation of colour, Goethe’s admirers would argue, can tell everything about its experience.

But Goethe’s attack was misplaced. Newton said his Opticks was about the mechanics of light, not the experience of colour. Furthermore, Goethe wrecked his phenomenological instincts with so many preconceptions – such as his view of archetypes and of the power of opposites – that no rigorous researcher today would start with it.

Still, Goethe’s polemics fascinate because he seems to champion a way of doing science that is different from the “usual sense”. Science, then and now, is often pictured as an activity that’s mostly about postulating “correct” mechanisms underneath phenomena, rather than about discerning phenomena in the first place. As the philosopher Ernst Cassirer wrote, “The mathematical formula strives to make the phenomena calculable, that of Goethe to make them visible.” The problem is that the first way threatens to drive out the second. Goethe inspired those who viewed his anti-reductive approach – colour is a human, not a spectroscopic, phenomenon – as rescuing not only the science of colour, but an entire way of being a scientist.

• Enjoy the rest of the April 2015 issue of Physics World in our digital magazine or via the Physics World app for any iOS or Android smartphone or tablet. Membership of the Institute of Physics required