Artificial Analysis Decision Index v2 Lifts Private Weight to 45%

Artificial Analysis shipped Decision Index v2 on November 8, lifting the share of the benchmark built on private datasets from 25% to 45% and introducing a headline Decision Score that rewards a model for "both correctness and calibrated confidence." The writeup frames the update plainly: the goal is "to better reflect real-world utility and guard against benchmark gaming."

The reshuffle rebalances almost every component. AA-Paperclip, a new private corpus of 1,500 documents including resumes, receipts, invoices, and research papers, lands at 30%. AA-Symmetry, which tests answer consistency across 120 questions with 440 variations, joins at 15% alongside a new EvalSafe-O*NET-AA slot at the same weight that plugs in TypeSafe AI's O*NET benchmark. AA-Omniscience-Decision v1.1 and HLE-MultiChoice v1.1 both fall from 25% to 15%, each now carrying ten "I don't know" phrasings. "We have retained When2Call from NVIDIA in the Index at a decreased weighting of 10% (down from 25%), and removed WildGuardTest given its high saturation," the article states.

At the top of the decision-model table, Drex Max v1.1 from Nace AI and Fastino Labs' GLiDE v1.2 tie at 73, with Baseten's Mammoth 27B at 72. On the language-model side, Claude Haiku 5.5 (Max) leads at 77 and GPT-6 Luna (Max) scores 76. Costs run from $14 to $859 per 1M tasks across decision models, with GPT-6 Luna (Max) at $1,530 — a spread that shows up alongside a similar widening in enterprise AI spend we have tracked across recent weeks. The Decision Score itself is defined as 50 × (1 + Accuracy − Brier score).