BottleCap AI's ThinkingCap Qwen fine-tune halves thinking tokens

The number in BottleCapAI's new ThinkingCap-Qwen3.6-27B release on Hugging Face worth staring at first is on GSM8K: the fine-tuned model reportedly uses 74.1% fewer thinking tokens than the base Qwen3.6-27B while accuracy actually climbs from 93.3% to 96.5%. Cheaper reasoning that is also slightly more accurate is not the trade-off the field has been assuming.

The broader claim on the model card is a 50% average reduction in thinking tokens, with macro numbers landing at 45.8% fewer tokens on out-of-domain benchmarks and 57.7% on the in-domain set the team trained toward. In practice it shows up unevenly. MMLU-Pro tokens drop 53.7% for essentially unchanged accuracy (85.9% to 85.4%), LiveCodeBench tokens drop 41.1% while accuracy rises from 80.7% to 84.3%, and safety-tuned refusal behavior holds within noise, with Nemotron-Safety moving from 98.9% to 99.0% and HEx-PHI from 99.9% to 100.0%. The card also reports out-of-domain truncation failures falling from 2.9% to 0.4%, which if it replicates is arguably the more useful production number than any single benchmark point.

Why this matters if you are not fine-tuning reasoning models yourself: the token bill for chain-of-thought inference has quietly become the dominant cost line for anyone running these systems at scale, and it is roughly proportional to how long the model "thinks." A recipe that halves that trace, if it survives outside the authors' own eval grid, changes the unit economics of reasoning-mode inference more than another accuracy point on a leaderboard does.

The honest caveats sit on the same page. These are BottleCapAI's own numbers on a self-published model card, not an independent replication, and the hardest reasoning benchmarks do slip: GPQA-Diamond drops from 85.5% to 83.8%, HMMT from 88.0% to 84.7%. What the card doesn't give you is the training recipe in any real detail, wall-clock latency rather than token counts, or behavior on long-horizon agentic tasks outside the listed suite. The interesting question for the next few weeks is whether the same "spend fewer tokens without losing much" property holds when someone else runs the model on their own workload.