This page records what was checked and what was found. Nothing has been removed from it, and the essay has not been edited to agree with it. The scope is facts only: dates, numbers, quotations, attributions, and whether a cited source says what it is cited for. The thesis and the prose were not assessed.
What was checked
Every number, quotation and attribution in the essay was compared with the sources it cites:
- Ioannidis JPA, Baas J, Klavans R, Boyack KW. A standardized citation metrics author database annotated for scientific field. PLOS Biology 2019;17(8):e3000384. doi:10.1371/journal.pbio.3000384
- Ioannidis JPA, Boyack KW, Baas J. Updated science-wide author databases of standardized citation indicators. PLOS Biology 2020;18(10):e3000918. doi:10.1371/journal.pbio.3000918
- Van Noorden R, Chawla DS. Hundreds of extreme self-citing scientists revealed in new database. Nature 2019;572:578-579 (the print headline is "Policing self-citations"). doi:10.1038/d41586-019-02479-7. The essay's second-hand source is Dockrill P, ScienceAlert, 29 August 2019: article.
- Moher D, Naudet F, Cristea IA, Miedema F, Ioannidis JPA, Goodman SN. Assessing scientists for hiring, promotion, and tenure. PLOS Biology 2018;16(3):e2004089. doi:10.1371/journal.pbio.2004089
- The HKUST research portal entry: Named among the "top two percent of scientists in the world"
- Stanford School of Medicine, launch announcement of METRICS, April 2014, and the 2005 PLOS Medicine essay.
The two database papers and the Moher paper were read as full text from the publisher. The Nature piece was read from the publisher's PDF. All links on this page were opened and load.
First: what held up
- The opening sentence. The essay says the 2019 paper opens with a warning. The abstract's first sentence is: "Citation metrics are widely used and misused." (The body text opens differently: "Use of citation metrics has become widespread but is fraught with difficulties.") The essay's wording is accurate for the abstract.
- The 2019 numbers. Six metrics; 22 fields; median self-citation 12.7%, full range 0.0% to 93.8%; 8,599 authors above 25%; 1,085 above 40%; 5,709 with a citations-to-citing-papers ratio above 2. The paper gives 176 subfields; the essay's "about 175" is fine.
- The caveats. Citations before 1996 are missing; whole-career metrics disadvantage young scientists; very small sub-subfields may differ; early-career researchers get little from any metric; "We discourage raw comparisons of scientists across very different fields"; "We cannot emphasize enough that use of these metrics needs to be prudent."
- The website. "Instead of creating a formulaic website, spreadsheets can be downloaded, searched, and tailored for analyses by scientists in whatever fashion they prefer." The essay's "explicitly choosing not to build a formulaic website" is the paper's own statement.
- The 2020 update. The addition of scientists in the top 2% of their subfield, the 4.9% figure and the peer-review manipulation sentence are all in the paper, as is "even within the same subdiscipline, different areas may still possess different citation densities, and assessing citation indicators always require caution."
- Nature 2019. The quotations are "I think that self-citation farms are far more common than we believe" and "Those with greater than 25% self-citation are not necessarily engaging in unethical behaviour, but closer scrutiny may be needed." The essay's paraphrase is faithful. ScienceAlert carries both.
- Other attributions. The Leiden Manifesto is reference 1 of the 2019 paper. Moher et al. 2018 is its reference 7, and Ioannidis is a co-author. METRICS was launched in 2014 under Ioannidis and Steven Goodman (Stanford calls them co-directors; "co-founded" is a fair description). The 2005 essay is real and correctly titled.
- HKUST. The portal lists the entry under "Prize: Honorary Award", "Granting Organisations: Stanford University", awarded Nov 2020. The essay's description is accurate for this example.
Second: what needs correcting or qualifying
1. "The word 'award' appears nowhere." This is not so for the 2019 paper. Its discussion of misuse refers to "other situations involving rewards (e.g., funding or awards)". The sense is different (decisions about awards, not a title the list confers), and the 2020 paper does not use the word, but the sentence as written is literally false. The essay itself later uses "awards" when summarising that passage. Safer wording: neither paper describes the database as an award.
2. "Ioannidis never called it a list of the world's best scientists." The 2019 abstract describes "a publicly available database of 100,000 top scientists", and the text calls them "the most-cited scientists". The authors do not say "best", but "top" is in their own abstract, and "never" is an absolute that cannot be checked against everything he has said or written. Safer: the papers describe a database of the most-cited scientists, not of the best.
3. The ratio of citations to citing papers is presented as a warning sign, without the authors' own balance. The essay calls it "a signal of citation stacking". The 2019 paper says high ratios "deserve more in-depth assessment", adds that "sometimes, this may reflect that it is common for a small number of papers of the same author to be cited together", and offers "citation farms" only as the alternative. The 2020 paper says such papers "may be fully legitimately often cited together". Nature also quotes him: the data "should not be used for verdicts such as deciding that too high self-citation equates to a bad scientist." A reader of the essay alone would take the ratio as stronger evidence of gaming than the authors do. "Citation stacking" is the essay's term, not theirs.
4. "What he called the egregious errors of raw bean-counting." The paper says the data should help "avoiding some of the egregious errors of raw bean-counting"; the essay drops "some of". Also, this and the other statements come from a four-author paper ("we"), written with Elsevier's Jeroen Baas, not from Ioannidis alone. The essay moves between "Ioannidis and his co-authors" and "he wrote" and "his closing line". That is a common compression, but "closing line" is inexact: the passage is in the closing paragraph, and the paper's last sentence invites readers to report errors.
5. The summary of the 2018 paper does not match what it says. The essay says it argued that institutions should reward "rigour, transparency and reproducibility, not counts". Moher et al. set out six principles: address societal needs; assess faculty on responsible indicators that reflect their full contribution; publish all research completely and transparently, whatever the results; reward open research; fund research on how to assess science; value out-of-the-box ideas. Transparency and open practice are there. "Rigour" is not a principle, and reproducibility appears in the background, not the recommendations. The 2019 paper cites it only for the "complex reward and incentive system of science". Suggested wording: it proposed six principles for assessing scientists, among them responsible indicators and rewarding complete, transparent publication.
6. "Citations before 1996 are not counted, which penalises older work." The first half is the paper's. The second half is the essay's inference. For papers from 1960 to 1995, citations received in 1996-2017 are counted and earlier ones are not, so the inference is reasonable, but the paper does not say it.
7. The database has moved on since 2020. The essay describes the 2019 and 2020 design in the present tense. Annual updates have followed, and the January 2025 PLOS Biology paper (Ioannidis, Pezzullo, Cristiano, Boccia, Baas, doi:10.1371/journal.pbio.3002999) added retraction data: of 217,097 top-cited scientists in the career-long list, 7,083 (3.3%) had at least one retraction, and the authors caution that retractions "do not always signify misconduct". Nothing in the essay becomes false, but a reader may assume the current release has no retraction field. One clause would cover it.
Third: not settled
- "Some universities record it ... as an honorary award." Confirmed for HKUST only. I did not survey other universities' profile pages.
- "Every autumn, scientists named in the ... list celebrate their inclusion with congratulatory posts and banners." A general observation, not tested. Release dates vary by year. The author revised this opening sentence on 8 October 2026, after publication.
- ScienceAlert. Read through a fetch tool, not in a browser. It carries the byline of Peter Dockrill and the date 29 August 2019; the essay's source list gives neither. The Nature article's online page redirects to a cookie gate, so the publisher's print PDF was used.
Originality scan
Eight of the essay's most distinctive sentences were searched as exact phrases (for example "The gap between the creator's caveats and the public celebration is the real story" and "a diagnostic instrument, not a hall of fame"). The essay's wording was also compared, by overlapping six-word runs, with the two database papers and the Nature article. Search engines do not strictly enforce quoted matches, so a zero is evidence of absence, not proof.
| Risk | Finding |
|---|---|
| HIGH | None. |
| MODERATE | None. The overlaps with the PLOS papers are the six-metric list, "given with and without self-citations", "small clusters of authors massively citing each other", "raw comparisons of scientists across very different fields", "a more nuanced use of metrics" and "the egregious errors of raw bean-counting". All are attributed in the text, and two are in quotation marks. One further phrase ("make citation metrics spurious and meaningless") is the paper's wording, attributed with "He wrote" but without quotation marks. Adding them would be tidier. |
| LOW | The sentences on technical problems (different databases, CV figures, erratic self-citation, field comparisons) follow the order of the 2019 introduction in new wording and with attribution. Standard formulations (c-score, six metrics, 22 fields) are shared factual scaffolding. |
| Searches | 8 of 8 distinctive sentences: 0 hits. Nature article: no six-word overlaps. ScienceAlert shares only the two Nature quotations. |
| Self-reuse | Searched the site's earlier essays and pages for Ioannidis, self-citation, h-index, "top 2%" and "bean-counting": no earlier piece covers this. No self-reuse. |
Minor points
The source list could add DOIs, the Nature article itself, and author and date for ScienceAlert. "The 2019 top 100,000" refers to the 1996-2017 data released in 2019. The composite indicator itself dates from Ioannidis, Klavans and Boyack, PLOS Biology 2016.
Limits of this check
Nothing was verified outside the sources named. Absolute claims such as "never called it" cannot be proved from a finite set of documents. The essay's interpretations (what the database is for, the closing argument) were not assessed.
Verdict
The essay is accurate on every figure and on its main quotations. Three sentences state more than the sources support: "award appears nowhere" (finding 1), "never called it ... best" (2), and the ratio as a mark of stacking (3). Finding 5 mischaracterises a cited paper. The rest are small. A reader can rely on the essay's account of the caveats; the summary of the 2018 paper should be treated with reserve.
© 2026 Husain Alkhaldy — fact-check of this essay.