Last February I was a panellist at a discussion on “Culture and metrics” at the Museum of Modern Art (MoMA) in New York City. The event was organized by Paola Antonelli, the museum’s senior curator of architecture and design and director of its new R&D department, which she founded last July. One of her goals is to identify ways to measure the museum’s impact on culture and the wider economy.

Antonelli, who originally studied economics before completing a Master’s in architecture, told me her idea for the department germinated after the 2008 financial crisis. “I had a chip on my shoulder,” she said. The crisis, in Antonelli’s view, revealed that traditional economic strategies were promoting investments that had a chaotic and even negative impact. Cultural institutions, on the other hand, offered “a slower but more dependable and effective key to long-term growth”. She felt sure that “more faith, attention and money would soon flow into cultural institutions”.

It didn’t happen. Disappointed, Antonelli sought to demonstrate to politicians and other prospective sponsors that cultural institutions exert a genuine and positive impact on growth. To do so, she recalled Lord Kelvin’s dictum: “When you can measure what you are speaking about, and express it in numbers, you know something about it; but when you cannot express it in numbers, your knowledge is of a meagre and unsatisfactory kind; it may be the beginning of knowledge, but you have scarcely, in your thoughts, advanced to the stage of science, whatever the matter may be.”

Measuring muddles

The traditional quantitative measures of museum productivity – foot traffic, membership size, number of exhibitions and so on – do not quite capture the spectrum of a museum’s influence. Antonelli therefore organized February’s panel to explore ways of demonstrating a more complete and long-term impact. About 200 people attended the event, which raised probing issues about measurement. One panellist was Kate Levin, the commissioner of New York City’s Department of Cultural Affairs, who presented statistics that impressively confirmed the importance of culture to city life. Her department estimated, for example, that The Gates – an environmental artwork erected by Christo and Jeanne-Claude in New York’s Central Park in 2005 – had attracted four million visitors and resulted in an estimated $254m economic boon for the city. She cited last year’s World Cities Culture Report, which concluded: “What links world cities to one another is trade, commerce and finance. What makes them different from one another is culture.”

Andrew Ross, a social and cultural analyst at New York University, mentioned factors that cause economists using traditional methods to underestimate the value of the arts. One is “cost disease” – the fact that productivity does not behave in the arts as it typically does in manufacturing. It takes, for example, the same number of people to perform a quartet today as in Beethoven’s time, though musicians’ real wages have increased. Another factor is “psychic income” – the willingness of artists to accept low wages in return for intangible benefits such as exposure.

I cited a distinction (originally presented in my 2011 book World in the Balance) between measurement against standards – what the SI is about – and measurement against ideals. The first is procedural and conventional, while the second is experiential and involves goals such as justice and education. The irony, I said, is that just when civilization is on the verge of perfecting measurement against standards with the “New SI” (which replaces artefact standards, such as the kilogram in Paris, with natural constants), measurement against ideals is more controversial than ever. We often pretend we can turn measurements against ideals into measurements against standards – by representing justice as a blindfolded woman holding scales, for instance. But this is metaphorical wish fulfilment: what could go in that pan? Small wonder that when we try to measure goals quantitatively it can give rise to front-page fights.

One obstacle to measuring against ideals is Goodhart’s law, a kind of Heisenberg uncertainty principle for culture. Named after the British economist Charles Goodhart, who devised it in 1975, the law essentially says that once a measure is chosen for making policy decisions, it begins to lose value as a measure. Goodhart applied it to banking policy, but in other fields, too, measurement can distort not only the practice being measured, but also perception of the goal. As soon as you measure intelligence, say, with standardized tests, schools begin to teach to the test – and you begin to view intelligence as a child’s ability to be taught to the test. If you measure researchers’ quality by the number of papers they produce, they start churning out unnecessary numbers of low-quality papers.

The critical point

Goodhart’s law tempts us to regard quantifying cultural impact as hopeless. We may recall with nostalgia the physicist Robert R Wilson’s famous testimony before Congress in 1969, when he was challenged to justify Fermilab’s new accelerator if it had no practical value even for defending the country. Wilson refused to manufacture a fake utilitarian reason and defended the device on cultural grounds. “[It] has nothing to do directly with defending our country,” he said, “except to help make it worth defending.”

But in today’s world, MoMA’s panellists agreed, Kelvin’s dictum prevails. Politicians, policy-makers and sponsors are measurement-driven, even with cultural matters. We therefore have to be more ingenious in devising metrics for cultural institutions. But to circumvent Goodhart’s law, we also have to recall that measurement involves not only an “it” – something measured – but also a “who” – the measurer. In measuring against an ideal, the measurer must not be anonymous; we have to be clear who is measuring and why.

As Antonelli put it: “The problem of measuring cultural impact cannot be resolved by numbers alone.”