Read the whole book free at the MIT Press
| Term | Meaning |
|---|---|
| Impact factor (Journal Impact Factor, JIF) | A score for a journal, roughly the average number of times its recent articles have been cited, published each year in the Journal Citation Reports. The editors argue that it predicts an article's impact rather than measuring it, yet it works like a price stamped on every article in the journal, and so creates a market (pp. 6–7). It was first published in 1975 to help librarians, and became disputed once it was used to judge researchers (p. 48). |
| Journal Citation Reports (JCR) | The yearly list of impact factors for the journals in the Web of Science. A journal dropped from it, as happened to journals that cited themselves too much, loses its ranking (pp. 10–11). Garfield began marketing it in 1972 (p. 35). |
| Web of Science | A database that records which papers cite which, descended from Garfield's Science Citation Index (p. 47). In the Czech case it served, together with Scopus, as a stand-in for international quality: being listed there counted as proof (p. 103). |
| Scopus | Elsevier's citation database, launched in 2004, which ended the Science Citation Index's monopoly (p. 47). De Rijcke and Stöckelová note that it listed some journals on Beall's blacklist while presenting itself as a full overview of world research (pp. 103, 106–107). |
| Google Scholar | A free search engine for academic writing that also counts citations. The Ike Antkare experiment showed that computer-generated papers could fool it, and the chapter concludes that it cannot be used to measure reputation (pp. 181–184). |
| h-index | A single number meant to sum up a researcher's citations. A researcher has an h-index of 10 if ten of their papers have each been cited at least ten times. Fanelli notes that it does not adjust for co-authors, which may tempt scientists to spread themselves across many joint projects (p. 118). The invented author Ike Antkare reached an h-index of 94 (p. 181). |
| Altmetrics | Measures of the online attention research gets, such as views, downloads, social-media shares, bookmarks and mentions on Wikipedia, as opposed to formal citations in other papers (pp. 214–215). Gingras finds them crude (pp. 50–51); Lin defends them and calls fears of gaming reasonable but premature (p. 216). |
| Goodhart's law | The rule that once a measure becomes a target, it stops being a good measure. The editors' examples: a school can massage what counts as a "successful student" in the figures reported about it, and a university can cap classes at nineteen students to meet the US News and World Report definition of a "small class" (p. 1). The editors argue that the new misconduct goes beyond this law (p. 1). Csiszar contrasts it with Merton's "goal displacement" and proposes a counter-law (pp. 37–39); Griesemer extends it to how causes and effects are structured (pp. 77–78). |
| Campbell's law | A related rule from a 1975 paper by Donald T. Campbell: the more a number is used to make decisions about people, the more it gets bent. The editors mention it, along with the economists' Lucas critique, as another name for the same idea (p. 18, n. 2). |
| Goal displacement | Merton's term for a means turning into an end in itself, as when the aim becomes collecting citations rather than doing good work. He warned Garfield in 1974 that citation counts would be manipulated in this way (pp. 33–34). |
| Audit culture / audit society | The habit of holding universities to account through numbers and checks, which the editors see as the setting for the new misconduct (p. 1). Power describes it as a "logic of auditability", an urge to produce paper trails whether or not anyone checks them (pp. 62–64). |
| Post-production manipulation | The editors' term for cheating aimed at the publishing process and at a paper's measured impact, not at its content: fake reviews, citation rings, hacked journal databases (pp. 2–3). |
| "Spammish" | The editors' word for practices that look like spam but in which the targets partly cooperate, so that those involved can call them legitimate or claim they were tricked (pp. 14–16). Lippman uses it in the same sense (p. 280, n. 3). |
| Citation cartel / citation ring | A group of journals or authors who agree to cite one another to push up their impact factors or personal counts. The Introduction describes eight Brazilian journals caught doing this in 2013 (p. 11). Brunton calls the top journals and universities an unofficial citation cartel (p. 248). |
| Coercive citation | An editor pressing authors, especially junior ones, to cite articles from the editor's own journal before a paper is accepted. The Introduction cites Wilhite and Fong (2012) for this (p. 10). |
| Self-citation (journal) | A journal's articles citing other articles in the same journal. Thomson Reuters dropped journals that did this too much, such as the Journal of Biomolecular Structural Dynamics (pp. 10–11). Sismondo describes a hidden version in ghost-managed papers, where a drug company's papers cite the company's other papers (pp. 130–131). |
| Predatory journal | The name the librarian Jeffrey Beall and others gave to journals that charge authors a fee and do little or no checking of the work (p. 9). The editors (p. 15) and de Rijcke and Stöckelová (p. 106) argue that the word misleads, since the authors are often willing customers, not "prey". |
| Beall's list | Jeffrey Beall's blacklist of predatory publishers and journals, kept from 2012 until it was shut down in January 2017 (p. 108, n. 6). |
| Hijacked journal | A fake website that poses as a real journal in order to collect authors' fees. Ike Antkare's chapter shows the case of Hermès and mentions a list of more than seventy (p. 186); Delfanti names Archives des Sciences and Wulfenia (p. 263). |
| Paper mill | A business that sells ready-made papers, or places on their author lists. The book does not use this term. It describes the related practice of selling authorship in papers already written and already accepted (p. 7), and COPE's finding of services that sold manuscripts and fake reviewer contacts (pp. 12–13). |
| Gift and ghost authorship | The book does not define these terms. Wager lists gift authorship among the questionable practices that metrics encourage (p. 229). In general use, gift authorship puts a name on a paper for someone who did not do the work, and ghost authorship hides the people who did. For the ghost side, Sismondo describes drug-company teams writing papers signed by respected academics, the "key opinion leaders" (pp. 123–127). |
| Ghost management | Sismondo's term for a drug company's partly hidden control over many stages of medical research: the research itself, the writing, the publishing and the spreading of the results (p. 123). |
| Key opinion leader (KOL) | A doctor or researcher whom drug companies value for their standing in a specialty, and recruit as an author or paid speaker (pp. 124, 128–129). |
| Fake peer review | Authors suggest reviewers for their own paper, but give email addresses they control, so they end up reviewing their own work. More than six hundred papers had been retracted for this (pp. 12, 145). |
| SCIgen | A program that produces nonsense computer-science papers, made in 2005 by three MIT graduate students to expose fake conferences, and later used by others in earnest (pp. 11–12, 178, 275–277). |
| SciDetect | Free, open software that spots computer-generated papers. Springer paid for it after it had to withdraw generated papers it had published (pp. 12, 188, 277). |
| The Ike Antkare experiment | Cyril Labbé's invented author ("I Can't Care"). About a hundred generated papers, all citing each other, gave him an h-index of 94 and twenty-first place among the most cited scientists on Scholarometer in 2010 (pp. 139, 181–184). |
| Retraction Watch | A blog started in August 2010 by Ivan Oransky and Adam Marcus that tells the stories behind retractions, the withdrawal of published papers. Its database went online in October 2018 (pp. 141, 144). |
| PubPeer | A website where readers discuss published papers, often without giving their names, run by the PubPeer Foundation (pp. 151–153). Lippman says Brandon Stell and George and Richard Smith founded it in 2012 (p. 273). Stell, a co-founder of the foundation, wrote chapter 11 with Barbour, a board member (pp. 285–286). |
| Post-publication peer review | Criticism of a paper after it is published, by readers rather than by the journal's chosen reviewers. PubPeer promotes it, and Brookes found that public criticism on a blog led to far more corrections and retractions than the same kind of problems left unpublicized (pp. 151–154, 174). |
| Impact Case Study (ICS) | A set-format story that UK universities wrote to show the social and economic benefit of their research in the 2014 national assessment, REF2014; 6,975 were submitted (pp. 58–60). |
| Responsible research metrics | The standard proposed in the report The Metric Tide: numbers should be sound, modest about what they show, open, varied and self-critical. Wouters argues it does not go far enough (pp. 71–73). |
| Contextualized judgment | Wouters's proposal to replace "informed peer review" with judgment that puts the context of the work first and does not set expert review against the numbers (p. 73). |
| Salami-slicing and "sandwiching" | Salami-slicing is cutting one set of results into many thin papers. Fanelli argues that scientists may instead be salami-slicing their collaborations, and "sandwiching" weak results inside long papers where they are hard to see (pp. 118–119). |
| Deficit model | Locke's term (2011), used by Kehm (p. 95), for the way rankings make every university except the top few look as if it is falling short (pp. 95, 99). |
| viXra | A preprint server (a site for posting papers without journal review) founded in 2009. It parodies arXiv's design, spells its name backward, and is open to anyone (pp. 261–266). |