arxiv.org web signal

ESMA: Evolution Strategy Trains LLMs to Know What They Know

By Published Updated

TL;DR

  • A paper proposes ESMA (Evolution Strategy for Metacognitive Alignment), a method aimed at making LLMs accurately report what they do and don't know.
  • The authors introduce a d'_type2 metric which, they say, isolates metacognitive ability from the biases and heuristics that confound existing evaluations.
  • They report ESMA's gains generalize across unseen datasets, languages, and newly acquired knowledge, and trace to a sparse set of parameters.

A new fine-tuning method called Evolution Strategy for Metacognitive Alignment (ESMA) teaches language models to know what they do and don't know, its authors say, with gains that generalize across unseen datasets, languages, and newly acquired knowledge.

Sangjun Park, Elliot Meyerson, Xin Qiu and Risto Miikkulainen pair ESMA with a measurement method built on the d'_type2 metric, framed as a way to isolate metacognitive ability from confounding biases. 'Evaluating true metacognition in Large Language Models (LLMs) is difficult due to biases and heuristics,' the paper opens. Parameter analysis, the abstract reports, 'reveals that these improvements are driven by a sparse set of parameters, offering new pathways for targeted metacognitive optimization.'

The abstract names no specific models tested, no benchmarks, and no numerical results. The paper was first submitted on 2 February 2026 and revised on 24 May; two researchers we track posted the link.

Shared on Bluesky by 2 AI experts

Originally reported by arxiv.org

Read the original article →

Original headline: Fine-Tuning Language Models to Know What They Know