Skip to main content
Policy and funding

Policy and funding

The real impact of impact

30 Jun 2011

With physicists under increasing pressure to prove their work has "impact", Mark Blamire warns it could force researchers to oversell and exaggerate their findings

Switched on

“Cool as a mountain stream” was the advertisement slogan for a certain brand of cigarettes from my childhood. It was written and paid for by companies that probably had a pretty good idea what smoking actually did to their customers. Such advertising was also supplemented by intense government lobbying – itself based on highly selective “research” – to oppose possible moves to restrict the sale and consumption of tobacco. The reason for all this? To preserve the company’s bottom line and secure the salaries and share options of the management. But that – ranging all the way from slightly disingenuous image presentation to almost outright deceit – is, after all, what advertisers do.

Scientists, of course, are different. In our laboratories we sift data to differentiate between rival hypotheses. We seek to push forward the boundaries of what is known and understood. We publish our findings so that results can be compared and, where discrepancies appear, further hypotheses can be advanced. And so the cycle continues with everyone constantly striving to increase our understanding of the universe around us.

Scientists are trained not be selective about data (at least not without good reason). Deleting or moving an “outlier” in a data set crosses an ethical boundary that the vast majority of scientists hold sacrosanct. There have been some exceptions: Jan Hendrik Schön, who was found guilty of 16 out of 24 charges of scientific misconduct in 2002, being a notable example in physics. However, once challenged, the scientific community tends to deal with the few cases of fraud rapidly and robustly.

The new problem, though, is that scientists – particularly those working in universities – are being increasingly urged to amplify the “impact” of their work. In many countries, including the UK, research funding is now becoming conditional on demonstrating impact criteria such as maximizing knowledge transfer to industry (to “secure our future prosperity”) as well as encouraging students to specialize in science. Giving an inspiring public lecture or publishing a paper reporting a genuine advance – the essence of research and teaching – are not enough anymore. That everything now also has to have “impact” means that scientific results are distorted, the significance of outcomes is overblown, and the presentation of the research becomes at least as important as the science itself.

Energy and funding

Impact has manifested itself in the “publish or perish” culture. The increasingly competitive marketplace of academic publication has placed paramount importance on journals’ “impact factor” – the number of citations in a particular year given to papers that appeared in the journal over the previous two years. To enhance this, many journals have an unwritten policy of preferring papers that are believed likely to attract more citations, and papers are also carefully written to inflate their broad significance.

But even this should be the least of our concerns. In the best journals, peer review is still at a high standard, and is normally performed objectively and honestly. Far more worrying is the funding environment in which all parties – the funding agencies, scientists and publishers – are complicit in cloaking scientific research with a far greater significance than can be objectively justified.

This is most obvious in the current focus on “energy” and the real danger of climate change. Governments have been persuaded to divert funding into energy research because it is a relatively inexpensive way of appearing to tackle the problem – at least in comparison with actually doing anything about it. Funding agencies then support it because the more funds they have to administer, the larger their operations become. Scientists also like it because we can use it to fund our research – most of which is only tenuously relevant to the reduction of energy consumption or its generation. Just as it has become possible to magically “offset” the carbon emissions associated with taking a flight or hiring a car, as a society we can offset our inactivity in actually reducing pollution by funding research that can be promoted as promising a greener future.

Of course, few research proposals actually promise perpetual-motion machines or teleportation devices. But they do promise to improve “efficiency” – the magic word for demonstrating the potential impact of such research. From new alloys for jet-engine turbine blades to LEDs for lighting, as less energy is required for a particular human activity, the new research is predicted to mean that a certain number of power stations can be shut down or so many million tonnes of carbon dioxide can be saved. But there is a catch. To governments and businesses, improved efficiency does not mean reducing use but rather the opposite – it increases the practicality and/or affordability of consumption. In the end, such developments oppose the stated impact of the research to reduce energy consumption.

Flatter the better?

Take flat-screen televisions as an example (liquid crystal, plasma, organic LED or whatever). These devices are probably intrinsically more efficient than the cathode-ray tube (CRT) that they have superseded – provided one compares the power consumption for the same screen size. However, one only has to visit an electronics retailer to realize that “efficiency” in this context is a meaningless term – modern televisions are simply enormous in comparison with their older counterparts. A quick visit to my local electronics store revealed screen sizes of up to 60 inches, and the power consumption of these mega-screens is correspondingly huge – more than 500 W in several cases, compared with a CRT television that requires about 50 W.

The key technological advance from the point of view of the manufacturers is not energy consumption but rather screen thickness. The depth of the enclosure of the cathode-ray tube had to scale with the width of the screen because electrons can only span a limited solid angle. However, a flat-screen television has no such restrictions and could cover most of a living room wall while still leaving room for the sofa. So, if we are being really honest, an accurate summary of the “impact” of the physics-led development of display technologies is that it has enabled televisions to have a much larger power consumption – an outcome that is unlikely ever to have appeared in a funding impact statement.

Another frequently trumpeted outcome of research is that a technology could become cheaper to manufacture. In the case of televisions, they are now cheap by historical standards, which has enabled people to purchase multiple sets and install them in more rooms in their homes and so increase power consumption still further. Indeed, this argument can be replicated across technology sectors: from colossally power-hungry data centres that support green-sounding “cloud computing” to cars where improvements in engine efficiency have been largely cancelled out by increasing size and mass of vehicles. So much for tackling climate change.

The moral high ground

It is about time that the scientific community applied the same intellectual rigour to the context in which it operates as it is supposed to apply to its own research. In other words, we should be able to justify objectively not just the chain of reasoning that has led to a scientific advance, but also any statements made regarding the potential impact of the result.

However, scientists simply do not have the commercial expertise or political insight to be able to do this. The recent allegations of fraud in the presentation of climate data illustrate the grave dangers inherent in trying to maximize the political “impact” of research. In reality, science can only directly contribute to tackling the major problems confronting our countries such as climate change and economic decline through educating the next generation about the science underlying current and future problems. Only then will they be able to take informed decisions themselves. As scientists, we will lose what remains of our authority and the moral high ground in that education debate if we continue to abandon scientific standards of proof in favour of spin and deceit in chasing funding.

Related events

Copyright © 2024 by IOP Publishing Ltd and individual contributors