Read the whole book free at the MIT Press ↗

The second part shows cheating on the numbers becoming a team effort. "Gaming" here means working the rules of a measure to get a better score without doing better work. The editors call the new pressure "impact or perish": not only to publish but to be cited and noticed. Under it, gaming spreads across the world and is carried out by groups and whole institutions, not just by individuals (p. 91). The editors give two examples. Some researchers put their names on papers written by a company's hired writers, so that they can "harvest" the citations. (A citation is one paper referring to another; counting citations is the most common way to measure a scientist's influence.) And some universities pay cash rewards for papers in top journals, to climb the league tables (p. 91).

Kehm explains why universities spend so much on rankings, and how a highly ranked university comes to stand for a nation's scientific strength (p. 91). De Rijcke and Stöckelová look at Europe's insistence on publishing "internationally". They show that it deepens a split between an "international" North and a "parochial" (narrowly local) South, and that this split can itself be gamed (p. 91). The editors add that universities are slow to punish staff whose papers in doubtful but "international" outlets improve the institution's figures (pp. 91–92). Fanelli argues that sharing authorship has become a group form of gaming. "Salami-slicing" usually means cutting one study into many thin papers; scientists now slice their collaborations instead of their results (p. 92). Sismondo describes a game with two players: drug companies supply papers they have quietly planned and steered from behind the scenes ("ghost-managed"), and academics supply their names (p. 92).

Chapter 6. Barbara M. Kehm, "Global University Rankings: Impacts and Applications"

Kehm opens with universities buying reputation. In December 2011 the journal Science reported that two Saudi Arabian universities were hiring "highly cited" researchers, scientists whose papers others refer to very often. The offer was about $70,000 a year for a short visit each year and for listing the Saudi university as an affiliation on their papers (p. 93). Within two to three years both universities went from not being listed at all to a place in the top 200–300 of the Shanghai ranking (ARWU), a widely watched annual league table of world universities (p. 93). In March 2012 the newspaper The Australian published a list of sixty of these "distinguished scientists", all of them men (p. 93). The same year, two Australian universities advertised jobs, at about $100,000 a year, for staff whose task was to manage the university's place in the rankings (p. 93).

Kehm sums it up this way: rankings "seduce and coerce" at the same time. They tempt and they force (pp. 94, 99). A university that joins the game has to take on its logic. Drawing on Morphew and Swanson, she lists what US universities do. They leave part-time teaching staff (adjuncts) out of their faculty counts. They present admissions figures so that the university looks hard to get into. And they pay for glossy brochures to lift their reputation scores (p. 94). By taking part, universities both question rankings and give them more authority (p. 94). In a 2007 survey by Ellen Hazelkorn, 93 per cent of university leaders wanted to rise in national rankings and 82 per cent in international ones (p. 94).

Yet moving up is hard. Between 1988 and 1998, twenty of the top twenty-five universities in the U.S. News ranking never dropped out of that group (p. 94). Global rankings cover only four to six per cent of the world's universities, so every university is judged by standards built for the very top. The result is a "deficit model": nobody except the leaders is ever good enough (pp. 94–95). Universities play anyway, for three reasons. A good position brings better students, donations and state money, so success breeds more success (the "Matthew effect": the rich get richer). Rankings hand out reputation. And politicians like them, because they make a complicated picture simple, and because top universities have become a sign of a nation's economic strength (p. 95).

Resistance has come from academics. Using Hazelkorn's figures, Kehm counts nine global rankings and more than fifty national ones (p. 95). Some universities ignore or boycott them, among them James Cook University in Australia, US colleges in the Annapolis Group and several Canadian universities (pp. 95–96). In Germany, learned societies called for a boycott of the CHE ranking, and four universities stopped sending it data, saying the work would need more than ten full-time staff (p. 96). In March 2013, three hundred economists protested against the ranking of professors published by the newspaper Handelsblatt (p. 96).

The last section looks at how rankings lead governments in different countries to steer policy the same way. The Shanghai ranking started as a tool for China's own use. But it created a worldwide story of competition between universities, and that story was then used to justify reforms. In Germany it helped bring about the "excellence initiative", and in Europe the funding of U-Multirank, another ranking (pp. 97–98). Kehm calls the result "postmodern": every government wants "one Harvard", and a place in a table comes loose from what universities actually do (p. 99).

Chapter 7. Sarah de Rijcke and Tereza Stöckelová, "Predatory Publishing and the Imperative of International Productivity: Feeding Off and Feeding Up the Dominant"

The chapter starts from a slogan of European research policy: "Be international!" (p. 101). In smaller countries such as the Netherlands and the Czech Republic, "international" is often taken to mean "good" without anyone asking why. International work counts for more than national work, and national for more than local (p. 101). The authors look at three things. The first is gaming metrics. The second is predatory publishing: journals that charge authors a fee and publish with little or no real checking. The third is the misuse of gold open access, the model in which the author pays so that a paper is free for anyone to read. They argue that all three can partly be seen as a logical response to the slogan "going wild" (p. 101).

Their case is a 2015 row at the Faculty of Social Sciences of Charles University in Prague. The background is the Czech system for judging research, set up in 2001, which awards "RIV points" for set types of output (p. 102). It was meant to be an objective and open machine. It became a tangle of weightings that even the officials who run it do not fully understand (p. 102). At the national level, almost nobody read the articles to judge them: there was hardly any peer review, the checking of work by other experts in the field. A listing in Web of Science or Scopus, the two big commercial databases of journals and citations, was taken as proof of international quality (p. 103).

A junior researcher who had earned his PhD in 2007 claimed seventeen books written or edited with others between 2011 and 2013, and more than eighty articles between 2006 and 2015 (p. 103). He was editor-in-chief, board member and even publisher of some of the "international" journals on his CV. Some of these journals were listed in Scopus, yet they also appeared on Beall's list, a blacklist of predatory publishers kept by the librarian Jeffrey Beall. Colleagues, the head of department among them, had written papers with him. And one co-author turned out not to exist, with a made-up post at Strasbourg and later at Cambridge (p. 103). The authors point out that he did exactly what the system asked, and did it skilfully. He even tried to build a new East–West "international" with himself at its centre (pp. 103–104).

His contract was ended in September 2015. The faculty issued rules warning against predatory journals and vanity presses, publishers that print a book if the author pays. Since 2010, more than twenty "international" books by its staff had come out with one such press (p. 104). A few weeks later the contract of the main whistle-blower, the person who had raised the alarm, was not renewed either, although many people backed him (p. 104). The faculty treated the case as one "bad apple" (p. 104).

The authors ask who benefits (cui bono). Their answer is that attacking predatory journals strengthens the hand of the big commercial publishers and their databases, which become the gatekeepers of what counts as "international" (p. 105). Predatory publishing copies the rewards that drive the mainstream industry. It sits at that industry's heart, not outside it (pp. 105–106). Its authors are hardly "prey": they game the system on purpose (p. 106). Scopus calls itself a complete overview of the world's research, but in practice it cannot check everything it vouches for (p. 107). Blacklists and codes of conduct assume there is one single science with shared rules. The authors end:

the more, the faster, and the more international need not always be the better. (p. 107)

Chapter 8. Daniele Fanelli, "Pressures to Publish: What Effects Do We See?"

Fanelli tests the most common story in this field: that pressure to publish pushes scientists to cut corners. He finds the story believable but poorly supported (p. 111). Surveys show what researchers believe, not what actually happens (p. 111). Retractions are rising. (A retraction is a journal's formal withdrawal of a published paper, usually because it is seriously wrong or dishonest.) But the rise mostly reflects the spread of retraction policies. In 2004 only 21 per cent of high-impact medical journals, the most cited ones, had such a policy; by 2014, 65 per cent did. And the number of retractions per retracting journal has stayed steady (p. 112). High-impact journals brought in these policies first, which explains why they retract more (p. 112). Studies showing that abstracts report more and more positive results are stronger evidence. But they are open to the "ecological fallacy": drawing conclusions about individuals from a trend across a whole group (p. 113).

The first of what he calls two blows to the story comes from his own work with colleagues. They compared 611 retracted papers and 2,226 corrected papers with 5,466 comparable papers chosen to match them (p. 114). The countries with the highest risk of retraction were those that paid cash for papers in high-impact journals, such as China, Turkey and Australia. The United States, where the pressure comes through careers and the race for tenure (a permanent post), was in the middle. The risk was lowest in the Netherlands and the United Kingdom, where funding follows national assessments and the pressure is felt most (p. 114). The most productive authors, and those publishing in top journals, were just as likely or less likely to have papers retracted (pp. 114–115). Two later studies pointed the same way: one of more than three thousand meta-analyses (studies that pool the results of many earlier studies), and one of papers with duplicated images (p. 115).

The second blow concerns productivity itself. Fanelli and Larivière followed 41,000 authors with over 760,000 papers between 1900 and 2013. They counted each paper "fractionally", sharing it out among its co-authors, so that a paper with four authors counts as a quarter of a paper for each. Counted that way, the number of papers per scientist in the first fifteen years of a career was flat or falling in most fields (pp. 115–116). The same held for papers on which the scientist was first author (p. 116). Today's CVs list more papers because scientists work together more (p. 117).

Fanelli says he is revising his own earlier view (p. 117). He does not call pressure a myth. But he calls it a "double mistake" to tie the rise in misconduct to rising pressure (p. 117). Instead, he suggests, scientists may be salami-slicing their collaborations: joining as many teams as possible and doing little in each. The reason is the h-index, a researcher's citation score: an h-index of 20 means 20 papers each cited at least 20 times. It gives full credit for a paper however many co-authors share it (p. 118). Papers have also grown longer, so negative results may now be buried inside them (pp. 118–119).

Rather than "salami-slicing" and selecting results, I suggest, scientists may be increasingly "sandwiching" them. (p. 119)

Chapter 9. Sergio Sismondo, "Ghost-Managing and Gaming Pharmaceutical Knowledge"

Sismondo describes where three economies meet: medical research, medical practice and drug marketing. The product is the medical journal article that looks like independent academic science but has been largely or wholly produced by companies, to influence what doctors prescribe. He calls this the "ghost management" of medical science (p. 123). His evidence comes mostly from industry conferences that he and two associates attended between 2007 and 2017. He does not name any of the speakers (p. 124).

Contract research organizations (CROs), firms paid to run studies, carry out most industry-funded clinical trials. Company statisticians analyse the data, medical writers draft the papers, and "publication planners" steer them into journals (p. 123). Authors are usually recruited after the paper is drafted. They are "key opinion leaders" (KOLs), chosen for their standing, because they agree with the conclusions, or as future speakers (p. 124). Planners map out key messages and respond to news with letters to the editor. They pick target journals by impact factor and by rejection rate (p. 125). (The impact factor is a journal's average number of citations per recent article, and it works as a rough sign of prestige.) In 1990, 70 per cent of industry research money went to universities and teaching hospitals; by 2000, 70 per cent went to CROs (p. 126). The plan for a drug expected to be a best-seller can include more than fifty articles over about three years (p. 126). The named academic authors give the article "a veneer", a thin outer coat, of independence, and rarely work with the data. One planner said half of companies show authors only the second-to-last draft (pp. 126–127). Editors know how this works, and papers are rarely if ever retracted for ghostwriting (pp. 127–128).

The same model extends to speaker programmes, in which companies pay doctors to give talks. One manager worried about explaining to investigators why a programme needed seven hundred to a thousand speakers. That number suggests some may be paid for prescribing rather than for speaking (p. 128).

Ghost-managed papers are also cited unusually often. A study of ninety-two known ghost-managed articles found them cited about ten times more than typical articles in the same journals; an earlier study of fifty-five found 2.4 to 2.9 times more (pp. 129–130). Sismondo gives three reasons. The articles have a lot of money behind them, so they are worth citing. Companies spread them widely. And plans of fifty or a hundred articles allow a hidden kind of self-citation: the articles cite each other, so it is a company, not a person, citing itself (pp. 130–131).

He refuses an easy split between fake and genuine science. Ghost-managed papers are often excellent medical science, built for marketing (p. 132). Misconduct is usually present, because the named authors do not meet the rules for authorship, but it is a different thing from faking data (p. 132).

pharmaceutical companies are now the biggest contributors to the evidence base of medicine (p. 132)