EdArXiv audit faults all 14 AI-in-education meta-analyses

A new preprint arguing that the quantitative evidence for AI in education does not hold up has gone up on EdArXiv. Its methodological audit of 14 meta-analyses, posted on August 12, 2026, concludes that none of them provided a valid basis for the claims they made.

The audit's wording is flat. In the audited studies, "none had a coherent construct, none sufficiently assessed publication bias, and all had severe heterogeneity." Statistics inside them were "misapplied, miscalculated, and misinterpreted." Primary study screening was "frequently problematic due to outcome measurement mismatches."

The problem is not individual sloppiness, the paper argues, but the shape of the evidence base. Meta-analysis in this field, it says, has been used to "manufacture certainty from an evidence base that is heterogeneous, positively selected, and weakly vetted." It attributes the pattern to "editorial and reviewer gatekeeping failures" across peer-reviewed journals rather than isolated author error.

The citations are the uncomfortable part. The 14 meta-analyses have accumulated "over 2,000 citations despite averaging only 16 months of publication," already woven into the literature long before the audit arrived. Three researchers we track shared the preprint in the days after it went up. The paper's remedy is more mundane than its diagnosis: mandatory data sharing and tougher editorial standards at education journals.