Editor’s note: This article was corrected on 19 February 2025, removing claims that 90% of papers are never cited and that as many as 50% are never read by anyone other than their authors, referees and journal editors. The claims were introduced by Physics World editors during editing and were not originally included in the article by the author.

It is a sobering fact that many papers that have been published in academic journals are never cited. Indeed, many papers are never read by anyone other than their authors, referees and journal editors. We know this thanks to citation analysis, a branch of information science in which researchers study the way articles in a scholarly field are accessed and referenced by others.

Citation analysis is, however, about much more than producing shock statistics. Along with peer review, it has over the past three decades been increasingly used to judge and quantify the importance of scientists and scientific research. Citation analysis is also the machinery behind journal “impact factors” – figures of merit that researchers take note of when deciding which journal to submit their work to so that it is read as widely as possible. Indeed, the output from citation studies is often the only way that non-specialists in governments and funding bodies – or even those in different scientific disciplines – can judge the importance of a piece of scientific research.

The Web has had a huge impact on citation-analysis research. Since the turn of the century, dozens of databases such as Scopus and Google Scholar have appeared, which allow the citation patterns of academic papers to be studied with unprecedented speed and ease. This could mark the beginning of the end for the 40-year monopoly of citation analysis held by the US-based firm Thomson Scientific, formerly known as the Institute for Scientific Information (ISI).

The ISI’s citation databases have always been criticized by scientists on the basis that they index a limited number of journal titles, that they cover mainly English-language titles from North America and Western Europe, and that they do not cover citations from books and most conference proceedings. However, the myriad of Web-based sources now provides a much more comprehensive coverage of the world’s literature, helping to usher in a new era of citation analysis based on multiple sources. Furthermore, the Web has led to several new citation measures and methods that were previously impractical, including article-download counts, link analysis, Google’s PageRank, Web citations and the “h-index” recently developed by US physicist Jorge Hirsch.

Out with the old

Citation analysis essentially involves counting the number of times a scientific paper or scientist is cited, and it works on the assumption that influential scientists and important works will be cited more frequently than others. Many governments, funding agencies (in the US at least) and tenure and promotion committees even use citation data to evaluate the quality of a researcher’s work, partly because they prefer not to rely on peer review and publication output alone.

However, not everybody thinks citation analysis is the best way to judge the validity of a scientific claim. Critics point to basic citing errors such as “homographs”, i.e. failing to separate citations to two unrelated scientists who happen to share the same last name and first initial. Cronyism, whereby friends or colleagues reciprocally cite each other to mutually build their citation counts, is another drawback. Other problems include people deliberately citing themselves or journals they are involved with; ceremonial citations, in which an author cites an authority in the field without ever having consulted the relevant work itself; and negative citations pointing out incorrect results.

Proponents of citation analysis, on the other hand, claim that these problems are relatively insignificant. Most citations found in articles and books, they say, are useful – by paying homage to pioneers, identifying original publications, providing background reading and alerting readers to forthcoming works. Citations also substantiate claims, give credit to related works and provide leads to poorly disseminated, poorly indexed or uncited works. According to Michael Koenig in the Palmer School of Library and Information Science at Long Island University in the US, citations provide – despite their ambiguities – “an objective measure of what is variously termed productivity, significance, quality, utility, influence, effectiveness, or impact of scientists and their scholarly products”.

The ISI citation databases – which include the Arts and Humanities Citation Index (A&HCI), Science Citation Index (SCI) and Social Sciences Citation Index (SSCI) – have for decades been used as a starting point and often as the only tools for conducting citation analyses. Since their original publication in the 1960s and 1970s these databases have grown dramatically in size and influence, and today contain about 40 million records from more than 8700 of the world’s most prestigious research journals. The SCI, which was launched in 1964, quickly became popular with scientists and librarians, and is now one of the most important multidisciplinary databases in the world.

Young researchers might find it hard to comprehend, but until 1988 these indexes existed only in print form, although searching them online has been possible since the mid 1970s using third-party information-retrieval systems such as Dialog. In 1988 the ISI supplemented its indexes with CD-ROM editions, and in 1997 the databases finally migrated online with the launch of Web of Science. The move to an online interface, which can analyse thousands of records in a few seconds, has given the ISI’s databases an even greater stranglehold in the field of citation analysis. But at the same time the Web has produced new publication venues and competitors that challenge the wisdom of continuing to use Web of Science exclusively.

Another problem with Web of Science is that it ignores the fact that scientists increasingly publish or “post” their papers online via open-access journals, personal homepages, e-print servers or in institutional repositories so that others can freely access the material. At the same time, researchers have started to search and download research materials via services such as arXiv.org, Google Scholar or publishers’ websites, like Elsevier’s ScienceDirect. Many of the millions of documents accessible via these services, which are published instantly to give the wider scientific community time to use and ultimately cite them, are not indexed by Web of Science. Moreover, an increasing number of Web-based services are enabling explicit citation searching (see box “Web-based citation-analysis tools”).

Web-based citation-analysis tools

The Web has given birth to more than 100 new databases or tools that allow citation searching. These fall into three categories.

The first allows the user to search in the full-text field to determine whether certain items, authors or journals have been cited in a document. Examples of these databases or tools include the following.

• arXiv e-print server (arXiv.org)
• CiteSeer (citeseer.ist.psu.edu)
• Google Book Search (books.google.com)
• Google Scholar (scholar.google.com)
• The Institute of Physics Journal Archive
• Physical Review Online Archive
• Elsevier’s Scirus (scirus.com)

Some of these tools, such as CiteSeer and Google Scholar, are based on autonomous citation indexing that allows automatic extraction and grouping of citations for online research documents.

The second category of databases or tools allows the user to search in the cited-references field to identify relevant citations. These tools first became available in the late 1990s when subject-specific databases began adding cited-references information to their own records. Examples include the following.

• NASA’s Astrophysics Data System Abstract Service (adsdoc.harvard.edu)
• MathSciNet from the American Mathematical Society (ams.org/mathscinet)
• Elsevier’s ScienceDirect (sciencedirect.com)
• SciFinder Scholar from the American Chemical Society (cas.org)
• Scitation/SPIN from the American Institute of Physics (pubs.aip.org)
• SPIRES-HEP at Stanford Linear Accelerator Center

The third category is databases that work exactly like Web of Science. The main and perhaps only good example of this category is Scopus (scopus.com), which was launched in 2004 by Elsevier. Although it covers more refereed journals and conference proceedings than Web of Science (15,000 titles compared with 8700) Scopus provides citation searching only from 1996 onwards, whereas Web of Science goes as far back as 1900.

Multiple citation sources

The rise in the use of Web-based databases and tools to access scientific literature has revealed how vital it is to use multiple citation sources to make accurate assessments of the impact and quality of scientists’ work. Take the book Quantum Computation and Quantum Information by M Nielsen and I Chuang (2000 Cambridge University Press), for example. According to Web of Science, this book has been cited more than 2800 times. However, Scopus says it has been cited 3150 times, Google Scholar 4300 times, Physical Review Online Archive 150 times, ScienceDirect 375 times, the Institute of Physics Journal Archive 290 times, and arXiv.org 325 times. If only Web of Science is used, we would miss all of the citations found through Google Book Search and arXiv.org plus hundreds of the citations found through the other databases or tools.

A citation study carried out recently by the present author and Kiduk Yang at Indiana University in the US is one of many that have shown the need to use multiple citation sources. We compared results of citation coverage from Web of Science, Scopus and Google Scholar for a sample of 25 highly published researchers in our field of information science and found that Scopus and Google Scholar increase the citation counts of scholars by an average of 35% and 160%, respectively. Perhaps more importantly, this increase varies considerably from one research area to another, with researchers working in computer-mediated communication and human–computer interaction having their number of citations more than doubled, while those specializing in bibliometrics and citation analysis had their number of citations increase by less than 25%.

Another major finding of our study is that the use of Scopus and Google Scholar has helped to establish a link between information-science research and cognitive science, computer science, education and engineering (as evidenced by the high number of citations from journal articles and conference papers in these fields). Such a finding about interdisciplinary trends in science could not have been uncovered by relying on Web of Science citations only. Indeed, multiple citation tools allow us to generate much more accurate maps or visualizations of scholarly communication networks in general, such as establishing links between authors, departments, disciplines, journals or countries that cite or influence each other.

While the emergence of comprehensive Web-based citation databases and tools – many of which have been around for only two years or so – has been received favourably by citation analysts, it has also made the job of searching and analysing citations more challenging. For instance, the new citation tools cover not only journal and conference papers, but also millions of unique items in various languages and forms, such as book chapters, dissertations, e-prints and research reports. This requires much more work than the relatively simple task of using Web of Science to compile and interpret citation searching and analyses based mainly on refereed journal articles.

For instance, we spent about 3000 hours collecting data from Google Scholar alone in order to carry out our recent study into the overlap and uniqueness between citation databases, compared with only 100 hours using Web of Science and 200 using Scopus for the same sample. Another consideration when performing citation analyses in the Web era is how to weigh up citations from journal versus non-journal sources and from refereed versus non-refereed sources. This is vital because citations from, say, the journal Nature Physics are of different quality and value than citations found in a master’s thesis that sits in a university repository.

The impact factor

When scientists seek research grants, file for tenure or promotion, or apply for staff or faculty positions, it has become customary to include both the impact-factor scores of the journals in which their papers were published and the number of citations received by these articles. As high-impact journals usually attract high-quality contributions from top scientists and have a large readership, publishing in these journals is a top priority for scientists who want to increase their visibility, prestige and influence among their peers; it also improves their chances of getting lucrative job offers and research grants.

The impact factor of a journal in a particular year is the number of citations received in the current year to articles published in the two preceding years divided by the number of articles published in the same two years. For example, Physical Review Letters has a 2005 impact factor of 7.489, which means that on average each of its 2003 and 2004 articles was cited 7.489 times in 2005.

The journal impact factor was launched by the Institute for Scientific Information (ISI) in 1975 and has been published annually since then. Initially, the impact factor was made available in microfiche format only, then ISI migrated it to CD-ROM in the late 1980s, before finally making a searchable version of it available on the Web in 1997. Currently, the ISI provides impact-factor data for over 5900 journals in science and technology and 1700 journals in the social sciences through the publication Journal Citation Reports. The impact factors always lag one year behind and are published each summer.

Quantifying your impact

The basic idea of using citation-based measures to assess the impact, importance or quality of a scientist’s overall work is to show how often and where he or she is cited. The two best known measures are citation counts and “impact factor” – the number of citations received in the current year to articles published in the two preceding years divided by the total number of articles published in the same two years (see box “The impact factor”). However, the fact that almost all scientific papers now exist online opens the door to other, possibly more accurate, citation-based measures.

The impact factor has several weaknesses. First, its scores can be significantly influenced by a few highly cited articles and/or too many uncited or low-cited articles. Second, authors and journals that frequently publish review articles tend to have their citation counts and impact exaggerated because these types of articles are usually highly cited. Third, citation counting and impact factors do not take into account articles that were used but did not get cited. Finally, the two-year “citation window” in the impact-factor formula fails to capture the “long-term value” or the real impact of many journals.

The Web has enabled a number of alternative citation-based measures to be devised to get round some of the limitations of the citation-counting and impact-factor methods. Of these, the most important are “download counts”, which became feasible only because of the migration to online publication, and the “h-index”, which exploits the rise of Web-based citation databases.

Using a download rather than citation count means that the impact of an article or a journal can be measured in real time, rather than having to wait several years after it has been published. According to Tim Brody and Steven Harnad in the School of Electronics and Computer Science at the University of Southampton in the UK, there is a strong, positive correlation between download counts and both citation counts and impact factors, although the degree of correlation varies from one research field to another. As downloads are instantly recorded and counted, Brody and Harnad suggest that the measure can be particularly useful for providing an early estimate of the probable citation impact of articles.

The h-index, meanwhile, was developed in 2005 by Jorge Hirsch, a condensed-matter physicist at the University of California in San Diego, to quantify the impact and quality of individual scientists’ research output (see box “How high is your h-index?”). The measure is simple: a scientist with an h-index of, say, 40 has published 40 articles that have each attracted at least 40 citations (papers with fewer than 40 citations therefore do not count). Hirsch estimates that after 20 years a “successful scientist” will have an h-index of 20, an “outstanding scientist” an h-index of 40, and a “truly unique” individual an h-index of 60. However, he points out that values of h will vary between different fields.

The h-index – which ranks Ed Witten of Princeton University top with an h value of 107 – immediately became popular among researchers because it captures the fact that scientists with very few high-impact articles or, alternatively, many low-impact articles will have a low h-index. The measure therefore helps distinguish between a “one-hit wonder” and an enduring performer who has numerous high-impact articles and hence a high h-index.

Moreover, a flurry of empirical studies conducted by librarians and others shows that the h-index correlates positively with citation counts, impact factors, publication counts and peer evaluation of research impact and quality. Finally, the h-index is very easy and quick to compute using databases or tools such as Web of Science, Scopus or Google Scholar. Indeed, almost an entire issue of the journal Scientometrics was recently devoted to the h-index, and the measure is now automatically calculated in the “citation report” function of Web of Science.

As with all citation-based measures, however, the h-index must be used with caution. This is because the index ignores, for example, why an item was cited in the first place; so negative citations to incorrect work are counted. Moreover, it is insensitive to highly cited works and disregards total citation counts. These last drawbacks have led to the development of two alternative indexes. The editor of Science Focus Jin Bihui has devised an “a-index”, which is defined as the average number of citations received by works in the number of h-index publications, while Leo Egghe from the Universiteit Hasselt in Belgium has devised a “g-index”, which is defined as the highest number, g, of papers that together received g² or more citations. (A researcher with a g-index of, say, 10 has published 10 papers that together have been cited at least 100 times.)

Indeed, Hirsch seems to have encouraged other physicists to develop their own productivity measures, with terms such as the “h–b index” and “creativity index, C_a” having appeared in preprints on the arXiv server in recent months. The h–b index was devised by Michael Banks from the Max-Planck Institute for Solid-State Physics in Stuttgart, Germany, to judge the impact of a particular field, whereas the creativity index was developed by José Soler of the Universidad Autonoma de Madrid in Spain to create and transmit scientific knowledge based on the network of citations among research articles. The creativity index concluded that Princeton University Nobel laureate Philip Anderson is the most creative physicist in the world (Physics World September 2006 p9).

Harnessing the potential

The citation databases, tools and citation methods mentioned here are just a few of many new and powerful indicators of research output that have become possible with the Web. Indeed, a search for articles in Web of Science reveals that the number of citation-based research-evaluation studies has been growing steadily over the years. Meanwhile, the exponential increase in the number of databases and tools that allow citation searching shows just how widespread the use and popularity of citation analysis has become. Funding agencies, as well as hiring and promotion committees, are increasingly relying on these methods to evaluate research to supplement other quality indicators such as peer review and publication output.

The Web has brought many changes and challenges to the field of citation analysis. Researchers and administrators who want to evaluate research impact and quality accurately will from now on have to use not only multiple sources – Web of Science and Scopus being the main two, but also Google Scholar, arXiv.org and others – but also different methods (e.g. citation counts as well as the h-index, and so on) to corroborate their findings. Relying exclusively on Web of Science and a single citation measure will, in many cases, no longer be an option for making accurate impact assessments.

Scientists now need to make it their job to disseminate their work on as many platforms and in as many different ways as possible, such as publishing in open-access and high-impact journals, and posting their work in institutional repositories, personal homepages and e-print servers, if they want their peers to be aware of, use and ultimately cite their work. Publishing a journal article is now only the first step in disseminating or communicating one’s work; the Web provides a multitude of methods and tools to publicize its scholarly worth.

More about: Citation analysis

C L Borgman and J Furner 2002 Scholarly communication and bibliometrics Ann. Rev. Info. Sci. Technol. 36 3–72

T S Brody et al. 2006 Earlier web usage statistics as predictors of later citation impact J. Am. Soc. Info. Sci. Technol. 57 1060–1072

L Egghe 2006 An improvement of the h-index: the g-index ISSI Newsletter 2(1) 8–9

J E Hirsch 2005 An index to quantify an individual’s scientific research output Proc. Natl Acad. Sci. USA 102 (46) 16569–16572

S Lawrence et al. 1999 Digital libraries and autonomous citation indexing Computer 32 (6) 67–71

M H MacRoberts and B R MacRoberts 1996 Problems of citation analysis Scientometrics 36 435–444

L I Meho and K Yang 2006 A new era in citation and bibliometric analyses: Web of Science, Scopus, and Google Scholar J. Am. Soc. Info. Sci. Technol. at press

H F Moed 2005 Citation Analysis in Research Evaluation (Springer, Dordrecht)

P O Seglen 1998 Citation rates and journal impact factors are not suitable for evaluation of research Acta Orthopaedica Scandinavica 69 224–229

Scientific publishing, like so many other aspects of human life, has been transformed by the Web. Almost all journals can now be read online and papers downloaded at the click of a mouse without ever having to visit a library. Indeed, the traditional commercial publishing model is being challenged by open-access preprint servers like arXiv.org, which – as a survey carried out for this special issue of Physics World reveals – is used by almost everyone in the physics community to access the latest research.

But while online access to papers is certainly convenient, and also makes scientific information more easily accessible to the developing world, it is little more than the digital equivalent of retrieving a paper from a traditional library. In other words, the information flow is still one way. Now, however, the Web is evolving. The next generation of the Web – a set of applications bundled under the loosely defined term “Web 2.0” – encourages people not just to use the Web as a reference source but also to interact with it.

Web 2.0 is all about making it easier for people to create and share content, ranging from digital photos of their cats to entries in user-edited encyclopedias. And it is undoubtedly big business, as demonstrated by News Corporation’s recent acquisition of the “social networking” site MySpace for $580m and by Google’s $1.6bn purchase of video-sharing site YouTube.

Given that the first incarnation of the Web was developed by researchers at CERN to aid their research, you might expect Web 2.0 to have a similarly revolutionary impact on the way physicists communicate and access information. For instance, some researchers are starting to use websites such as “blogs” and “wikis” in their professional lives. But as our survey reveals, rather than being the vanguard of these developments, this time round, physicists may be getting left behind.

Battles in the blogosphere

By far the most widespread manifestation of user-generated content on the Web is “blogging”. A blog (short for “web log”) is essentially an online diary, featuring regular entries, or “posts”, by one person or a small group. Anyone reading the blog can add a written comment on a post, creating lively debates between the denizens of the so-called blogosphere. Since the term was first coined in 1997, blogs have become an Internet phenomenon – 60 million are now listed in the blog directory Technorati. With so many people voicing their opinions, the content is often banal. But blogs have also been lauded as a new form of “citizen journalism”, for example providing us with eyewitness accounts of dramatic events such as 9/11.

Physicists on Web 2.0

Several science magazines and academic journals have set up blogs, featuring rolling reports from conferences or updates on the latest science news. In addition, many professional physicists have blogs of their own. Some discuss research at an academic level; some see their blog as a form of public outreach, providing plain English explanations of the latest physics stories; and others use blogs as a forum to debunk bad science reporting in the media. (A new column in Physics World looking at popular physics blogs begins this month (see “Blog life: Uncertain Principles”).

In the Quantum Diaries project, 33 physicists blogged about their life and work to celebrate World Year of Physics in 2005; and 40 more scientists run blogs as part of the popular ScienceBlogs portal. If it seems like too much of an effort to keep track of so many frequently updated webpages, the “aggregator” site Mixed States compiles the recent updates from all of the major physics blogs in one place.

While some may see them as vanity projects, physics blogs are starting to have a real impact on the way researchers communicate. For instance, several papers have already been published on arXiv.org that cite blog entries, demonstrating that blogs are becoming a bona fide channel for scientific communication. Meanwhile, a debate about string theory that began in the blogosphere has recently been thrust into the spotlight, being widely reported in science magazines and picked up by national newspapers.

Cloud of knowledge

The most outspoken critic of string theory, Columbia University mathematician Peter Woit, has used his blog Not Even Wrong to point out that string theory does not make predictions that can be tested by experiment, and that the status granted to the theory as the most promising approach to reconciling quantum physics with gravity diverts resources away from other alternatives.

His blog has sparked a fierce, and sometimes unexpectedly personal, debate, both in the comments posted on Not Even Wrong and on string theorists’ blogs such as Luboš Motl’s Reference Frame and Clifford Johnson’s Asymptotia. Such slanging matches may not be typical of the level of discussion in physics departments, and cause one to wonder how much of the debate depends on the easy anonymity that such online forums offer. But, for better or worse, blogs have opened up a new form of discourse in physics that can – as it is carried out in such a public fashion – be propelled into a broader context in a way that a discussion at a conference, say, would not have been.

In fact, the media attention generated by the “string wars” has already claimed one blogger as a casualty. Christine Dantas, a Brazilian physicist, regularly discussed the problem of quantum gravity in her blog Background Independence. Then in November she closed the blog down, explaining on an online forum that the media coverage of the string wars had made her uncomfortable, especially after her blog was mentioned in a Brazilian physics journal. “I am a quiet person, and wish to go back to my quiet life, to my quiet readings and studies,” she wrote.

In this case, however, has the very nature of blogging inflated obscure disagreements between small groups of physicists into something that looks like a “holy war”? The intricacies of quantum gravity have no impact on the work of the vast majority of physicists, but they occupy a disproportionate amount of the discussion in the physics blogosphere. Some have compared the blog community to an “echo chamber”, where people tend to write about things they have seen on other blogs. For instance, Sabine Hossenfelder, co-author of the blog Backreaction, says that viewing the string debate through the eyes of blogs is “like putting magnifying glasses on a spot on your nose and then getting obsessed about it”.

Taking a step back from such passionate debates, blogs are also influencing the careers of the physicists who write them, though it remains to be seen whether blogs are perceived as a positive example of outreach work or a waste of time that would be better spent on research. California Institute of Technology cosmologist Sean Carroll, who contributes to the most popular physics blog Cosmic Variance, has certainly raised his profile through blogging. He is now seen by the mainstream media as someone who they can rely on for accessible explanations of difficult science (see “Blogging for physics”).

But how many physicists actually read or contribute to blogs? While 16 of the 60 respondents to our survey said they do read physics blogs, all but three of these are the very same people who also write blogs of their own. The Mixed States site lists about 100 physics blogs, which is a drop in the ocean of a worldwide community of hundreds of thousands of physicists. Indeed, most of our survey respondents were either unaware of the existence of physics blogs or said that they do not trust the content of them. “I ignore blogs completely,” says theoretical particle physicist Frank Close from Oxford University. “I wouldn’t read what someone posts on a notice board outside my local newsagent and putting it on the Web doesn’t make it any more official.”

Wikifying the Web

Trust is a recurring issue in the online world. A few years ago the idea that one of the most-consulted sources of information in the world would be an online encyclopedia that can be modified by anyone who uses it would have seemed ridiculous. But that is exactly what has happened with Wikipedia, probably the most well known of the Web 2.0 sites. As it has turned out, the self-correction that is built in to the system has worked pretty well – barring occasional controversies, erroneous modifications are usually quickly corrected by another user. Indeed, a study carried out by Nature in 2005 suggested that, for science, Wikipedia is almost as accurate as the Encyclopaedia Britannica, with the average Wikipedia entry containing around four inaccuracies to Britannica‘s three.

From the Big Bang to quantum computing, there is a wealth of physics information on Wikipedia, though detail is sparse on more obscure topics. According to our survey, most physicists now seem happy to use Wikipedia as a quick reference; indeed, 75% of our respondents say they consult Wikipedia for physics information. The level of trust physicists have in the encyclopedia varies markedly, however, though all agree that anything crucial should be cross-checked with the original source. Harvard string theorist Motl thinks that the quality of articles in Wikipedia is high, “especially in sufficiently general topics that have been edited, verified and refined by a large number of editors”. But some remain unconvinced. “I wouldn’t dream of reading Wikipedia for physics,” says Nobel laureate Philip Anderson. “Nor would I trust it if I did.”

In fact, Wikipedia is the most successful example of the general concept of a user-editable webpage, or “wiki”, which is being put to use by physicists for other purposes. In a large collaboration, a wiki can be a great way to get round the problem of disseminating tacit knowledge and “tricks of the trade”, especially when collaborators are spread over the world and cannot take part in coffee-room discussions. Such wikis already exist in experimental-particle-physics collaborations like CDF at Fermilab and ATLAS at CERN, for example, and are gradually evolving into comprehensive repositories of information about these experiments.

“I can well imagine that blogs and wikis will become the framework for brainstorming and discussing ideas,” says Gordon Watts of the University of Washington in Seattle. “It may even end up in some cases that ideas are fully formed on blogs and never make it into a preprint, let alone peer review.” In fact, some have suggested that a framework based on blogs and wikis could be the basis of a new type of peer review. In this model, a paper under review would be posted on a public webpage where comments could be appended. The text of the paper itself might even become a wiki that could be edited.

A precursor of this idea was the decision by the administrators of arXiv.org to allow “trackbacks” to papers posted on its server. Trackbacks are a crucial feature of Web 2.0. While the original “hyperlinks” of the Web are one-way, simply pointing from one site to another, trackbacks are a means of notifying a webpage that another webpage has added a link to it. In the case of arXiv.org, if a physicist writes about a particular paper in their blog and sends a trackback request, that blog entry is then automatically included on the paper’s page.

Such a combination of arXiv preprints and blog comments has already proved fruitful. For example, physicists Robert Alicki, Daniel Lidar and Paolo Zanardi produced a revised version of their paper on quantum error correction (arXiv.org/abs/quant-ph/0506201) in the light of discussions on Dave Bacon’s blog The Quantum Pontiff. Bacon, who works on quantum computing at the University of Washington, was naturally delighted. “This is now my favourite comment on an arXiv paper,” he posted to his blog.

However, such a system is not without its problems. Discussions on physics blogs are often derailed by crackpots promoting their own off-the-wall theories or political opinions. In an attempt to keep such distracting elements out of arXiv‘s system, all trackbacks are moderated by its eight-member physics advisory board. But this raises its own questions, such as who should and should not be allowed to add a trackback. Last year, for example, Woit was told that he would be excluded from the system because he did not meet arXiv‘s definition of an “active researcher”. Hotly disputing what he saw as a slur on his reputation, Woit accused Jacques Distler, a string theorist who sits on the advisory board, of bearing a personal grudge against him. He remains excluded.

Socializing online

Despite being at the forefront of the development of the Web in the 1990s, physicists have been slow to embrace some of the innovations offered by Web 2.0 . “Social tagging”, for example, is a form of classification by users that is widely used in sites such as the photo-sharing forum flickr. Users choose “tags” to describe their photos – “me”, “London”, “red” and so on – and can then search on not only their own tags, but also those of millions of other users. This “bottom-up” form of classification can also be used for scientific information. Indeed, the sites Connotea and CiteULike were set up specifically to apply social tagging to scientific papers.

The idea is that when you find a useful paper online, you save it in your account and add tags describing the content of the paper. If this was just a way for individuals to manage their references, it would not be terribly exciting. The innovative aspect of sites like Connotea is the social side: you can see what other users are saving and search their tags to look for new papers on a topic of your choice. However, biologists seem to have adopted the concept much more eagerly than physicists – with “genetics” and “metabolism” being among the most popular tags on Connotea (see figure) – and only one of the respondents to our survey has so far used such sites.

In the wider world, there is no doubt about the most popular Web 2.0 destination. MySpace is a “social networking” site that allows each of its users – a staggering 120 million, mostly teenagers – to produce a personal homepage with photos and details of their likes and dislikes. The page also prominently displays how many “friends” the user has on the site, and friends can add comments to each other’s pages, creating something of an online popularity contest. This might seem an unlikely place to find a physicist, but that would be reckoning without string theorist Michio Kaku, author of popular-science books Hyperspace and Parallel Worlds. Kaku, who lists himself as married/straight/non-smoker/non-drinker, has accumulated an impressive 2725 friends on his page.

There could, however, be a more serious use of social networks for physicists. Jennifer Golbeck, a computer scientist at the University of Maryland, has found that a social network like MySpace contains useful information about who knows whom and how much people trust each of their contacts. Golbeck is working on algorithms to use this information to tell you how much you should trust someone you do not know, based on their position in your social network. This trust measure could then be used by scientists and collaborations. For example, you might allow access to an early version of a paper only to people whose trust rating is above a certain threshold.

It is clear that a brave new world of blogs and trackbacks, wiki peer review, social tagging and trust networks awaits physicists who dare to venture into Web 2.0. But while physicists are happy to publish and download papers online, there is a certain pining for print. “Undeniably, the Web offers several important advantages,” says 85-year-old Nobel laureate Jack Steinberger, “but it also has some unfortunate drawbacks, and I am nostalgic for the good old days when publishing something was a more serious event in our lives than putting something on the Web.” Perhaps Web 2.0 will not fully make its mark on physics until the MySpace generation hits the labs.

Linking up with the social Web

Blogs by professional physicists are becoming increasingly popular. Those mentioned in the article include the following.
• Asymptotia (asymptotia.com)
• Backreaction (backreaction.blogspot.com)
• Cosmic Variance (cosmicvariance.com)
• Not Even Wrong (www.math.columbia.edu/~woit/wordpress)
• The Quantum Pontiff (dabacon.org/pontiff)
• Reference Frame (motls.blogspot.com)

Many others can be found at the physics blog “aggregator” Mixed States (mixedstates.something similar.com), or at the ScienceBlogs portal (scienceblogs.com). Technorati is a comprehensive directory of all blogs (technorati.com). The sites CiteULike (citeulike.org) and Connotea (connotea.org) allow scientists to store and search for information about papers using “social tagging”. Jennifer Golbeck’s research on trust in social networks can be found at trust.mindswap.org and Michio Kaku’s MySpace page at www.myspace.com/mkaku.

Rüdiger Voss extols the virtues of open access

The scientific community has welcomed the idea of open access to the research literature through the Internet with open arms. Various initiatives, statements and declarations in recent years have all recommended free access to scientific results through self-archiving, the creation of new open-access journals and the conversion of subscription journals into open-access publications. Increasing numbers of funding agencies even force their grant holders to make their papers freely available online.

Physicists have always been at the forefront of the open-access revolution, which seeks to disseminate research findings as widely as possible. They, after all, pioneered the preprint system, which involves making copies of papers available to one’s colleagues while the work is still being peer reviewed by a journal. With the advent of the Web, particle physicists in particular were quick to create electronic versions of the system through preprint servers such as arXiv.org or the SPIRES database. Indeed, more than 90% of the research literature in particle physics is now freely available on the Web, which has replaced traditional journals as the lifeblood of scientific communication.

However, publishing papers via open-access Internet databases – rather than in reputable open-access journals – has been a mixed blessing. If an advance copy of almost every journal paper in a particular field is freely available online, libraries are more likely to cancel subscriptions in these self-archiving fields than in those where the practice is not as prevalent. What this means is that an increasing number of researchers – some in prestigious universities – can no longer read important journals in particle physics and related fields. A rift is fast developing, in which fewer and fewer scientists have access to the final, peer-reviewed version of a paper, while the rest have to make do with a preprint that is rarely identical to the final published version.

If journal publishers continue to hike subscriptions well above the rate of inflation in the face of declining circulation, journals will eventually cost so much that only a small number of major libraries and institutions will be able to afford them. Obviously, this business model is not sustainable for publishers in the long term and there is a big risk that it could collapse. Nonetheless, researchers are still as keen to publish their work in established journals like Physical Review Letters as they were in the pre-Internet age – after all, refereed journals are vital for career progression in academia, given the status that peer review confers on a scientific publication.

But should the physics community pay for expensive journals that merely rubber-stamp definitive versions of research papers, which cannot be found in many libraries, and that have lost their original raison d’être, which is lively and active scientific communication? The answer is, quite simply, no. Open access to the final, peer-reviewed version of scientific literature is the only way out of the dilemma. It will give back to refereed journals the role that they played 20 years ago and let them live in peaceful co-existence – but on an equal footing – with institutional repositories.

However, high-quality open-access publishing comes at a price. If subscriptions are abolished, open-access journals will probably have to be supported by charging authors a fee to publish each paper or by some form of sponsorship that could go as far as research agencies funding entire journals. Critics of open access argue that these scenarios are risky for two reasons. First, they say, author fees encourage publishers to lower quality standards and acceptance thresholds in order to maximize revenue. Second, this funding model is unstable: why would a researcher pay to put their work in open-access journal X when they could just as well publish it for free in equally prestigious journal Y?

I do not believe that first concern is serious: the principal reason why the traditional publishing paradigm has served us so well in the past is that it has relied on independent editorial boards. In this respect, there is no difference between subscription and open-access journals. The second concern is more serious, but it can be overcome by a large-scale transition to open access, of the type that CERN is promoting, accompanied by a programme to help scientists realize the true cost of the publication process.

Open-access journals will benefit all sides involved in the publication process. Scientists will profit from barrier-free access to research results. Authors will profit from increased dissemination of their work. Funding agencies will profit from more transparent, competitive and cost-effective pricing; while publishers will profit from a source of income that is rooted directly in the scientific community concerned, and will be more reliable than the increasingly fragile system of a dwindling number of overpriced subscriptions.

In particle physics, many publishers have understood the potential benefits and have started giving readers free access to individual articles in a journal by charging those particular authors a fee. Such “hybrid” business models can be useful to initiate a smooth transition to open access, but full open access to entire journals must remain the ultimate goal. The transition to open access is not without risk; but if the risk is well managed and has the support of all parties concerned, then there are far fewer dangers than sticking with obsolete – and increasingly absurd – subscription schemes.

John Enderby advocates caution

John Enderby

Much is made by advocates of open-access publishing of the notion that our human rights include, in some sense, “the right to know”. However, in his recent book The Access Principle (2005 MIT Press), John Willinsky from the University of British Columbia makes the crucial distinction between “open” access and “free” access. Most moral philosophers would argue that there is a hierarchy of rights with perhaps clean water, food, clothing and shelter at the top. But none of these is free.

In many societies this apparent contradiction is resolved by forcing those who can pay for food and shelter to do so, while providing welfare payments to those who cannot pay. Once it is recognized that access to reliable information and the right to know likewise have a cost, the question arises as to who should pay for the necessary validation and dissemination.

It is at this point that the disagreements arise. In its purest form, open-access publishing would offer all material in its final, edited, formatted and paginated form freely available, with the publication costs being entirely borne by the authors of papers or the people who funded their work. The traditional “subscription” model, again its purest form, makes material accessible only to those who pay for it, with authors paying nothing towards publication. Between these two extremes is a continuum of business models.

I have great difficulty with open access in its purest form. Economic models in which the producer pays – but the consumer does not – are, to say the least, unusual. At the moment, if researchers do not like a particular journal, they can choose to publish elsewhere. But if all journals were open access, consumers would not be able to exercise any influence over the market. Instead, presumably, the funding agencies would have the upper hand, having to decide how much of their resources would go to publication costs.

And here we meet another difficulty. There is no universal figure for the cost of publishing research papers because it depends strongly on the proportion of papers that are rejected. Publishing research papers is unusual in the business sense because a lot of time, energy and money goes into dealing with papers that do not meet the quality threshold of the journal in question and so do not appear as a “product”. Most commentators now agree that the costs in the quality end of the market are about £1500–£2000 per published paper.

Some advocates of open access have talked about charging researchers a certain amount when they submit a paper and then making them pay an additional fee if their paper is published. However, this approach is bureaucratic and open to abuse. Imagine sending a paper and cash to a publisher and then having the paper rejected. Could you then ask your funding agency for more money so you can submit the paper again and, if so, for how long could this continue?

I am also worried about the implication for developing countries. If author charges became the norm, there may be pressure from aid agencies for scientists from these nations to publish their work in less prestigious, low-impact journals that charge less because their acceptance rates are high. At present, all authors can have their research reviewed free of charge in any journal of their choice. Open-access publishing could therefore lead to journals being dominated entirely by scientists from the richest nations.

And finally those countries with an active scientific workforce would be out of pocket in two ways. Researchers in the UK, for example, produce about 75,000 papers a year, which means they would have to pay about £100m in author fees if all journals were open access. This sum is far higher than the £90m they currently pay in library subscriptions.

Second, a lot of high-quality research in Europe is published in US-based journals. In other words, if all journals were open access, hard cash from research budgets would end up in the coffers of American publishers, although this would be partly offset by a saving on subscriptions. The loss of income from journal subscriptions overseas could also threaten learned societies like the Institute of Physics and the Royal Society.

I do, however, have some reservations about the subscription model in its purest from. As a trustee of the International Network for the Availability of Scientific Publications, which seeks to make papers available to developing nations, I am aware of the problems of making people pay for information. Thankfully, the open-access debate has forced publishers to tackle some of the disadvantages of the subscription model.

Many publishers now give readers free access to all articles either for a limited period following publication or once a certain time has elapsed. Others are experimenting with hybrid models in which authors can choose to pay a publication charge in exchange for open access. Most publishers also now allow authors to post the accepted versions of their papers online.

My view is that market forces will lead to variety of models. However, for us all to move to open-access publishing, which is a so far unproved business model, is not in the best interests of science until experimentation has revealed some of its unintended consequences. I am therefore uneasy about governments or anyone else imposing new rules on authors as these could lead to unforeseen distortions in the market. It must be for each scientific community to decide for itself how best to organize the publication of its research.