A few months ago the New York Times used the curious term "black elephant", attributing it to the London-based investor and environmentalist Adam Sweidan. The term crosses two already familiar expressions. One is "black swan" – a name for something whose repercussions force you to throw out key theories that you took for granted, such as the premise that "all swans are white". The other term is "elephant in the room" – something whose presence everyone knows but nobody seriously addresses out of fear or embarrassment.

Sweidan's examples of "black elephants" were environmental and included global warming, ocean acidification and pollution of water supplies. To address these issues on the required scale, Sweidan said, would profoundly disrupt current political activity. So we ignore them. A black elephant is therefore something that changes everything, but which no-one wants to deal with.

I believe black elephants are found in pure science as well. In January, for example, I wrote about shutdown ceremonies for synchrotron radiation sources. I noted that such mundane, even cliquey, events are natural and spring from the special character of the scientific community and the feelings of its participants. Yet you won't find these events mentioned in books on how science works, nor could anybody I spoke to could clearly describe their value.

I had "captured it well", e-mailed one scientist, mentioning his mixed feelings of pride and gratitude, affection and loss at the closing ceremony for a machine he had worked on. His remark convinced me that, in shutdown ceremonies, I had spotted a black elephant. In science, in other words, a black elephant is a familiar feature that is routinely excluded from formal accounts of how science works, and could not be incorporated without ruining them.

Why some elephants are black

The scientific world is a sprawling and untidy place whose inhabitants practise their craft in myriad ways. Attempts are periodically made to bring order to this world by building model homes in it, so to speak, and declaring that what's inside is what science is really like – all the activities outside being imperfect versions. That way, we can easily teach it and tell outsiders what it's about.

Two such homes are particularly attention-grabbing. The first is orderly, its atmosphere logical, and its disputes calmly resolved by proposing theories and taking data. Experiments are good when they get the true result, wrong when they don't. This house does not have normal people inside – the inhabitants are so exacting and rule-abiding that they live and act quite differently from the rest of us. Discoveries made inside this house are universal, reflecting truths about nature outside. This house was built by traditional philosophy of science.

Another house was erected in reaction to the first. Its inhabitants behave exactly as non-scientists do, motivated by the same social and psychological forces. Experiments are good when they get a result everyone accepts. What's found in the room is not universal but local – arising from what's happening in that room. Obtaining consensus about a result is a matter of swapping interests, like the work of diplomats. This home, built by "social constructivists", has real people inside but no real nature.

My characterization is simplified, and each model house has undergone modifications. Still, it's a good first approximation. The model houses might be different, but they have in common that they seek to give an abstract, formalized representation of the scientific process from the perspective of someone outside the territory. They differ in what they include and omit. The first, to oversimplify, gets rid of human beings, who disrupt the rationality inside the house. The second gets rid of nature, which would resist, define and frustrate the negotiations.

Each house has a different set of black elephants, common and easily recognizable features of the scientific life that it would be impolitic to discuss. In the first house, one black elephant is the example I mentioned – shutdown ceremonies at machines that are integral to the lives and work of communities of scientists. In the second house, however, a shutdown ceremony is not a black elephant; such events might be read, for instance, as deliberate attempts to consolidate and reinforce a community in preparation for a political drive to request a new facility.

The cost of house-building is illustrated by recent silly debates about whether string theory can truly be considered science if it has no testable predictions. String theory doesn't fit into the first model house. But the scandal here is not about the flawed character of string theory as science, but about the flawed architecture of the first house. The scandal should even cause us to examine our fascination for house-building.

The critical point

"We learn how to do science," Steven Weinberg writes in his new book To Explain the World, "not by making rules about how to do science, but from the experience of doing science, driven by desire for the pleasure we get when our methods succeed in explaining something."

Science is a way of making sense of the world and has gone on for centuries. It has continually accreted knowledge during that time and is always being extended further, but – as Weinberg says – not in a form that can be reduced to rules or negotiating interests. If you are trying to describe science – rather than just do it – you have to beware of your assumptions, including why you are looking at it and what you are hoping to find; you also have to be ready to revise them when necessary.

So in describing how science works we shouldn't aim at building permanent houses. Instead, we should create temporary tents that let us watch science in action – ready to be taken down and moved. That way we can appreciate the elephants, rather than paint them black.