nature.com web signal

Nature Reviews Psychology: proprietary LLMs break replication

By Published Updated

TL;DR

  • Nature Reviews Psychology argues proprietary LLMs' hidden architecture, training data and parameters make psychology studies impossible to reproduce.
  • The authors recommend studies using proprietary APIs include at least one open-source baseline alongside the closed model.
  • Unannounced provider updates to model versions and system prompts can silently break replication months after a paper publishes.

Nature Reviews Psychology has published the kind of methods comment reviewers tend to cite for years. Four authors from Princeton's psychology department argue that if your study ran on a closed LLM API, no one else can rerun it.

Killian L. McLoughlin, Alexis Palmer, J. Nathan Matias and M. J. Crockett write that "the use of proprietary models poses risks to transparency and reproducibility." Their fix is standardizing reporting practices and prioritizing open models.

Their specific complaints are concrete. Proprietary systems are black boxes whose architectures, training data and model parameters are not publicly available. GPT-4, the authors note, has withheld details about its "architecture, pre-training corpora, dataset composition, and parameter count." On top of that, "unannounced version and system-prompt updates from providers can alter model behavior between runs," so the same prompt a paper's authors used can return something different to a replicator months later.

The practical ask is modest: studies using proprietary APIs should include at least one open-source baseline.

Shared on Bluesky by 4 AI experts

Originally reported by nature.com

Read the original article →

Original headline: Proprietary LLMs impede scientific transparency and reproducibility