Are you seeing a blotch on your microscope slide, or a cell? Have you discovered a new astronomical object or is it just light bouncing off a support structure in your telescope? Is your clock telling you that neutrinos are travelling faster than the speed of light, or do you have a loose connection between your GPS receiver and the clock?

There’s no more common, or fundamental, question in experimental science than whether what you are looking at is an artefact or a signal. “The cornerstone of experimental knowledge,” writes the Virginia Tech philosopher of science Deborah Mayo in Error and the Growth of Experimental Knowledge, “is the ability to discriminate backgrounds: signal from noise, real effect from artefact.”

Trickster

Like many others, I met my first artefact in a high-school physics lab. I had followed the lesson plan carefully, but my results indicated that what I had measured didn’t agree with Coulomb’s law. I took several readings, and was careful to measure and re-measure the distance between the two charged conductive spheres that were repelling each other, and the amount of twist in the torsion wire suspending one of them. My strange result had staying power.

Like many others, I met my first artefact in a high-school physics lab.

So did I think I had discovered a fifth force of nature? No. I figured that I must have screwed up. Why was I sure? Because I knew better from the teacher, the textbook and the results of other students in the class. But I couldn’t figure it out. The teacher came over and after a few puzzled moments realized that the standard-issue power supply was faulty.

That was my first artefact. My experimental set-up had tricked me. It looked like it was telling me something about nature, but instead it was telling me about itself. The teacher used my mishap as the occasion for an admonition about how to avoid getting artefacts. Measure everything twice. Double-check your equipment. Substitute parts if you can. Don’t necessarily trust the factory-built elements.

Researchers, though, have to judge whether what they have is an artefact or signal without the benefit of teachers like that; they are trying to come up with what will appear in the next generation of textbooks, and know that experiments to follow will show whether their work is right or not. These researchers are susceptible to what I think of as “experimentalist’s anxiety”.

I came away from my high-school lab with the notion that an experiment is like a little performance. You collect a set of props or “performers”, make sure you understand them and what you do, set them to working together, and if you set them up just right their performance shows the audience something they didn’t know before. But if you don’t set the performers up just right, the performance isn’t very good and they tell you nothing new.

Offbeat and tantalizing

Defining an artefact is easy – it’s something that your instrument is showing you with no counterpart in reality; it’s produced by your equipment or your techniques rather than by nature. I think there are at least two kinds of artefacts. The kind that haunted me I’d call a “klutzy” artefact – caused by your misunderstanding or overlooking some behaviour in the equipment. But there’s another kind that I’d call a “tantalizing” artefact – produced by something truly novel in the experiment’s performance that you can’t quite make out.

An example of such an artefact – one that involved the “real” ghost of a fifth force – occurred at Brookhaven National Laboratory in 1961, just a few years before my experience in the high-school physics lab. At Brookhaven, a group led by Yale University physicist Robert Adair accelerated bunches of protons, smashed them into a steel target, used electromagnetic fields and other means to sweep away everything but long-lived neutral kaon particles. Known as “K-longs”, these particles were directed into a bubble chamber where their decays could be imaged.

The experimentalists rebuilt and improved every part of the equipment to try to eliminate the impossible two-pion decays, but could not.

The results showed an impossible number of two-pion decays, at least according to a fundamental part of the prevailing theory known as CP symmetry. The experimentalists rebuilt and improved every part of the equipment to try to eliminate the impossible two-pion decays, but could not. The only thing the group members could not rule out was a fifth force of nature, which they hinted at in their published paper.

Another group from Princeton University, whose leaders – James Cronin and Val Fitch – had the office next to Adair’s, noted the baffling result, mounted an experiment structured so that the performance would show whether CP symmetry was violated or not, and led the audience to conclude that it was. That conclusion earned Cronin and Fitch the Nobel Prize for Physics in 1980.

No doubts were ever expressed about the quality of the Yale experiment. Yet its result was ambiguous, and it was only the Princeton experiment that cleared things up. By staging an experiment to look just at the role of CP symmetry in K-long decays, the Princeton experiment changed the structure of elementary particle theory, and was able to “parse” the Yale result, showing what part was due to CP violation.

But because the Yale result had only a hint of CP violation, mixed in with other things, some science historians have insisted it was an artefact. “It was an artefact,” declared the US physicist and philosopher Alan Franklin. “A spurious result stimulated the work of the Princeton group.” But since the ambiguity of the Yale result was at least partly due to CP violation, it was at least a tantalizing artefact.

The critical point

But experimentalists must have encountered more types of artefacts than klutzy and tantalizing. Let me know what artefacts you’ve run in to and I’ll write up your amusing – or disastrous – experiences in a future column. Let’s hope the collective experiences of Physics World readers will help you avoid mistakes of your own.