Found first: a primary source the press has not covered yet.
A new open benchmark called Quant-Arena tested nine frontier language models on end-to-end quantitative trading research, from feature construction to live-cost evaluation on Chinese A-share data. GPT-6 Astra scored highest among all nine, and no model beat the equal-weight market return in the sealed 2024 test year. The paper is available on arXiv.
What the source says
The benchmark, described in "Quant-Arena: Benchmarking LLM Agents for Quantitative Trading Research" by Houde Qian, Zheng Jiang, Wenyi Li, and nine co-authors (no affiliations listed in the paper), gives each model the task of constructing stock-scoring features and writing trading strategy code on 2022-2023 A-share training data. The resulting strategies are frozen and run through a sealed 2024 evaluation that incorporates realistic trading costs and market impact modeled from a simulated order book. Final scores weight realized excess return most heavily, supplemented by signal quality and the diversity of features the model used. GPT-6 Astra finished first across all nine models, and no model crossed the equal-weight market return in the held-out 2024 period.
Why it matters
Finance companies are spending heavily on LLM-based trading tools. Most published evaluations omit real trading costs and out-of-sample data. Quant-Arena applies both, and even with the best available model, no strategy cleared a passive equal-weight portfolio on Chinese A-shares in 2024. The benchmark is open to new submissions.