Nature Reviews Psychology: proprietary LLMs break replication

There is a version of AI-in-science where behavioral researchers stop worrying about which model they used and cite GPT-4 like any other tool. A Nature Reviews Psychology piece argues that version is not compatible with reproducibility, because the models behind those citations are moving targets that nobody outside the lab that built them can actually inspect.

The core complaint is straightforward. Proprietary LLMs are black boxes: their architectures, training data, and model parameters are not publicly available. The article calls out GPT-4 specifically, noting that OpenAI withholds details about its architecture, pre-training corpora, dataset composition, and parameter count. External researchers cannot access the hyperparameter configurations or optimization strategies that produced a result, so they cannot fully redo what an author did even with the same prompts.

The situation gets worse across time. Providers push unannounced version and system-prompt updates that can alter agent behavior between runs, meaning a prompt that gave one result to a paper's original authors can give a different result to a replicator months later. On top of that, the article flags a data-leakage concern: since it is unclear what data a proprietary model was trained on, a benchmark or stimulus set may already sit in the model's weights and quietly inflate whatever it is measuring.

The recommendation is what you would expect. Prioritize open-weight models, which let researchers deploy the identical model version by publishing and making trained parameters freely available for download, and standardize reporting so studies that rely on a proprietary API include at least one open-source baseline. The reasoning is that open-weight evaluations reportedly produce more stable and replicable findings than API-based evaluations of equivalent proprietary systems.

The honest caveat is that this is a perspective in a review journal, not a large audit. The retrieved material does not quantify how many published psychology findings would actually fail replication under the proprietary-LLM regime, nor how big the capability gap is between the best open-weight model and the leading API today. But the direction is the useful part. Journals, preprint servers, and funders that want their AI-assisted psychology corpus to still be defensible in five years have a very clear ask sitting in front of them: require the open baseline.