Google's ToolGrad flips tool-use data pipeline, ties Gemini 2.5 Pro

Google researchers say a 12-billion-parameter open model fine-tuned on synthetic data can match Gemini 2.5 Pro at tool calling. The system that produced the data, described in a Google Research blog post, reverses the usual pipeline: instead of writing a synthetic user query and then hunting for API calls that satisfy it, ToolGrad drafts a working tool-use solution first, then writes the query to match.

The pipeline runs four modules in sequence. An API Proposer narrows candidates, API Executors test them in parallel, an API Selector picks the top-performing call as what the authors call a "textual gradient," and an LLM Updater rewrites the synthetic user query and AI response. The post reports a 99.8% pass rate in data generation and fewer optimization steps than baseline query-first approaches.

On the Berkeley Function Calling Leaderboard, ToolGrad-12B scored 83.1, effectively tying gemini-2.5-pro at 83.2 and, per the post, coming in ahead of claude-4.5 Opus at 82.8 and gpt-5 at 74.4. Consistent gains showed up across Gemma-3 1B, 4B, and 12B fine-tunes trained on a small dataset the authors call ToolGrad-500. The API pool is drawn from ToolBench, which the post describes as containing more than 16,000 real-world APIs.

The authors, Google XR's Zhongyi Zhou and Ruofei Du, argue models trained this way "outperform their teacher models," pointing to a self-evolving loop. Two researchers on our radar shared the paper the day it went up. The comparison points are BFCL scores; the post publishes no per-cost figures and no multi-turn agent numbers, and the underlying APIs come from ToolBench rather than a live enterprise stack.