Leike and Hutter show AIXI optimality depends on UTM choice
TL;DR
- AIXI, the universally intelligent agent proposed by Hutter in 2005, has no known invariance theorem for the choice of universal Turing machine.
- Leike and Hutter show adversarial UTM choices make AIXI misbehave drastically, undermining its existing optimality properties.
- In the class of all computable environments, every policy is Pareto optimal, making Legg-Hutter intelligence entirely subjective.
Every policy is Pareto optimal in the class of all computable environments. That is the headline negative result from Jan Leike and Marcus Hutter's COLT 2015 paper on AIXI, the universally intelligent agent Hutter proposed in 2005.
"Our results are entirely negative," the abstract states. For Kolmogorov complexity and Solomonoff induction, the authors note, invariance theorems hold: the choice of the universal Turing machine shifts bounds only by a constant. For AIXI, no such theorem is known, and the pair show the gap is not benign. "Unlucky or adversarial choices of the UTM cause AIXI to misbehave drastically," they write.
The second blow lands on measurement. Legg-Hutter intelligence, and with it balanced Pareto optimality, "is entirely subjective," the paper argues. Leike and Hutter stop short of discarding the framework: "While it may still serve as a gold standard for AI, our results imply that AIXI is a relative theory, dependent on the choice of the UTM."
Submitted on October 16, 2015, the paper has resurfaced in AGI-theory conversations; two researchers on our radar flagged it recently.
Shared on Bluesky by 2 AI experts
-
anyway, if you're interested in Hutterian models of AI, here's Hutter (2015) and his paper about why the MIRI model of the Bad Robot doesn't work arxiv.org/abs/1510.04931
View on Bluesky →
Originally reported by arxiv.org
Read the original article →Original headline: Bad Universal Priors and Notions of Optimality