Jev Decision Index ranks 30+ open-weight decision models

The Jev Decision Index, a leaderboard posted to Hugging Face by multimodalart, scores decision models against a frozen suite of 132,422 requests across 37 benchmarks.

Nineteen of those benchmarks form the scored panel, grouped into five equal-weight areas: Tools & Automation, Retrieval & Classification, Language Understanding, Knowledge & Reasoning, and Arts & Human Judgment. The headline metric is 100 times the mean of the area scores, on a 0-100 scale.

The house rules are what make the leaderboard useful. Engines can't truncate requests, can't drop options, and get one fixed prompt rendering across the suite. "Unsupported, errored, abstained and pending requests score zero before chance normalization," per the suite documentation. A common 442 questions are dropped for every engine at scoring time.

According to HuggingNews, the release ranks Jev alongside more than 30 open-weights decision models, framed as "asking 130,000 questions to each participant to measure their effectiveness in automation, knowledge, understanding, and creativity." Two of the researchers on our AI Weekly radar posted the link the day it went up.

Models have to submit via pull request with a dataset link and hardware specifications, and entries are re-scored during review.