GroupTravelBench tests LLM agents on group travel plans
TL;DR
- GroupTravelBench sets up 650 group travel-planning tasks across three difficulty levels, using real user profiles, POI data, and ticket prices.
- Across frontier models the authors tested, plan validity stayed below 12%, and every agent fell short on all four rule-based outcome metrics.
- The benchmark targets three group-specific skills: eliciting private preferences via dialogue, coordinating inter-user conflicts, and balancing group utility against fairness.
Even the strongest frontier LLM agents produce valid group travel plans less than 12% of the time, according to a new benchmark posted by Xiang Cheng and colleagues on arxiv.org.
GroupTravelBench comprises 650 tasks across three difficulty levels, built from real user profiles, POI data, and ticket prices, run inside what the authors describe as a 'synchronous group-chat sandbox with cached tool data for reproducible offline evaluation.' It probes three capabilities single-user benchmarks skip: eliciting private preferences through dialogue, coordinating inter-user conflicts via compromise or subgrouping, and planning that balances group utility against fairness.
The paper argues that existing single-user travel benchmarks are 'approaching saturation,' and reports that even the strongest agents 'fall short on all four rule-based outcome metrics.' The abstract publishes no per-model scores and names none of the frontier systems tested.
Shared on Bluesky by 2 AI experts
Originally reported by arxiv.org
Read the original article →Original headline: GroupTravelBench: Benchmarking LLM Agents on Multi-Person Travel Planning