Paper2Agent turns 74 of 100 bio papers into working AI agents
TL;DR
- Paper2Agent agentified 74 of 100 computational biology papers, with 593 of 599 generated tools passing automated validation.
- Its agents scored 91.2% on 300 benchmark questions versus 80.3% for a Claude Code baseline (P < 0.0001).
- Three chained paper agents pinpointed GPR137 as the probable causal gene for psoriasis variant rs887314 in CD4+ T cells.
Stanford's Jiacheng Miao and colleagues report in Nature that they turned 74 of 100 computational biology papers into working AI agents using a system called Paper2Agent, which wraps each paper's code, data and workflows behind natural-language interfaces built on Model Context Protocol servers.
The pipeline crawls the paper's repository, sets up the environment, extracts callable tools from the tutorials, and validates each one. Across the 100-paper sweep, 593 of 599 proposed tools passed automated validation, and the resulting agents scored 91.2 ± 1.6% on 300 benchmark questions versus 80.3% for a Claude Code baseline (P < 0.0001). An agent built from the AlphaGenome paper produced 22 tools in about 45 minutes for $14 and hit 98.7 ± 1.3% accuracy on tutorial queries.
The framing is the point. "Conventional research papers require readers to understand and adapt the paper's code, data and methods to their work, creating barriers to dissemination and reuse," the authors write, arguing that agents make a manuscript "active, agent-native knowledge rather than static text." Five researchers we follow in our Who's Who tracker flagged the paper.
They also demonstrate cross-paper collaboration: three agents chained together implicated GPR137 as the probable causal gene for a psoriasis-associated variant, rs887314, in CD4+ T cells, with a Spearman correlation of 0.613 between predicted and measured effects (P = 3.79 × 10−3).
The ceiling is on the page. Twenty-six of the 100 repositories could not be agentified because of missing code, data, or environment failures, and the authors concede that "Paper agents require ongoing maintenance as upstream codebases and dependencies evolve." Their benchmarks, they add, measure "faithful execution rather than of analytical validity."
Shared on Bluesky by 5 AI experts
-
It would be fun to listen to a future episode of the Mystery AI Hype Theater 3000 podcast (that uses *ridicule as praxis") about this article 🤣 "Reimagining research papers as interactive and reliable AI agents" www.nat…
View on Bluesky →
Originally reported by nature.com
Read the original article →Original headline: Reimagining research papers as interactive and reliable AI agents - Nature