Language models don't reliably encode the current year

Prompting a language model with today's date updates the answer it gives to "what year is it?" 94.6% of the time. It updates the year implied by the model's verb tense 1.7% of the time.

That gap sits at the center of a new arXiv paper by Suze van Adrichem, Aditi Bhaskar, Diyi Yang, Christopher Potts and Jing Huang, a paper two experts in our Who's Who directory have already shared. The authors build two probes for the current year: a "declarative" task that asks the model directly, and an "associative" task that infers a year from verb tense. Both tasks, they write, "estimate current years within one year of the post-training data cutoff of instruction-tuned language models." Across 13 base models, the associative probe lands within an average error of only 10 months of the pre-training cutoff.

The problem: the two probes reach different mechanisms inside the model. "The associative task uses mechanisms similar to factual recall, while the declarative task lacks consistent causal pathways," the authors report.

Prompting a target year hits 94.6% success across 351 target years on the declarative task and 1.7% on the associative one. Year-shifted supervised fine-tuning matched the target on the associative side in only one of eight models. Weight editing works on each task alone but does not carry across both.

The paper's blunt summary: "the current year is not consistently encoded in language models."