Fake AI Citations Slip Into ENISA and White House Policy Papers
TL;DR
- Between May 2025 and April 2026, ENISA, South Africa's draft AI policy, and a US health assessment all cited research that did not exist.
- A 2023 Scientific Reports study found 55 percent of GPT-3.5 citations and 18 percent of GPT-4 citations were entirely fabricated.
- A 2025 analysis estimated at least 13.5 percent of 2024 abstracts had been processed with a language model, up to 40 percent in some fields.
Three official documents, three sets of citations that turned out not to be real. Between May 2025 and April 2026 a European cybersecurity agency, a South African national AI strategy, and a flagship White House health report all released work built partly on research that does not exist, according to Rachel A. George, a Stanford lecturer in international relations, writing in Tech Policy Press.
The pattern is what makes this more than an editing story. ENISA's threat landscape reporting contained 26 incorrect footnotes out of 492 by one count. South Africa withdrew its draft national AI policy after media reporting found at least six of its sixty-seven academic citations were fabricated. The US Make Our Children Healthy Again assessment cited studies that did not exist and misattributed others. Any one of these is embarrassing on its own. All three inside a year, from bodies with real budgets and oversight roles, is a signal that the underlying tooling is leaking fake references into work that shapes national decisions.
George's argument is that this belongs in the national security conversation, not the research-integrity one, and the base rates make the case. A 2023 study in Scientific Reports found that 55 percent of GPT-3.5 citations and 18 percent of GPT-4 citations were entirely fabricated when the models were asked to generate literature reviews. A 2025 analysis estimated that at least 13.5 percent of 2024 abstracts had been processed with a language model, rising toward 40 percent in some fields. Meanwhile more than 10,000 papers were retracted in 2023, a record driven largely by industrialized paper mills that sell fraudulent manuscripts. The drafting layer and the source layer are both getting noisier at once.
The honest caveat is that these are three high-profile cases and the piece does not tell you how many similar documents were audited and came back clean, or exactly how the fabricated citations in each case were introduced. It also does not give a concrete playbook for who pays for verification inside a small policy office, or how disclosure rules would be enforced when a national AI policy is drafted with tools built elsewhere. Those are the load-bearing details that decide whether this becomes a solved problem or a recurring one.
The forward-looking read is that citation verification starts to look like procurement infrastructure. Agencies that require retrievable-source configurations, mandate AI disclosure on submitted drafts, and fund independent checks will produce work that survives adversarial scrutiny. Everyone else will keep finding their next flagship report picked apart by outside researchers with a couple of hours and a browser.
Shared on Bluesky by 3 AI experts
-
Policymakers treat critical infrastructure as physical: power grids, communications networks, transportation systems, and satellites. In the age of generative AI, writes Rachel A. George, another kind deserves equal atte…
View on Bluesky →
Originally reported by techpolicy.press
Read the original article →Original headline: The Next National Security Challenge Is Research Integrity