arxiv.org web signal

WildSEEK finds a third of real-user queries are high-risk

By Published Updated

TL;DR

  • WildSEEK compiles 3,000 manually annotated information-seeking queries from real user interactions, covering risk-sensitive domains like health and finance.
  • Over one-third of the queries are classified as high-risk, with analytical queries appearing more often in that category than factoid ones.
  • Tested LLM responses show four recurring failures: sycophancy, overreliance, a default US-centric perspective, and poor handling of vulnerable populations.

Three thousand information-seeking queries, pulled from real user interactions and manually annotated for risk, make up WildSEEK, a new benchmark described in a paper accepted at EMNLP Main 2026. Over one-third of those queries land in high-risk territory. Analytical questions show up in that bucket more often than factoids do.

The paper, by Tanise Ceron, Joachim Baumann, Elisa Bassignana, Berat Cabuk, Dirk Hovy and Debora Nozza, focuses on risk-sensitive domains including health and finance. The authors also run their trained classifiers across more than 1.8 million realistic user queries to see how risk distributes at scale.

The failure story is specific. The paper reports that language model responses break down in four recurring ways: 'sycophantic behavior, overreliance, a default US-centric perspective, and poor handling of vulnerable populations.' Failure rates on analytical queries ran substantially higher than on factoid ones.

The abstract summary does not name which models were tested or publish per-model numbers. Two researchers we track shared the paper to our Who's Who feed after it went up.

Shared on Bluesky by 2 AI experts

Originally reported by arxiv.org

Read the original article →

Original headline: WildSEEK: Evaluating Language Models for Information-Seeking