Thrilled to share our latest collaboration with Google DeepMind: Formal Conjectures, an evolving Lean 4 benchmark for evaluating AI capabilities in formal math. arxiv.org/abs/2605.13171
- The Formal Conjectures benchmark contains 2,615 mathematical problem statements formalized in Lean 4, sourced from areas of active mathematical research.
- It splits into 1,029 open research conjectures for zero-contamination proof discovery and 836 solved problems for autoformalization testing.
- The dataset is released under Apache 2.0, and the authors report it has already been used to resolve open research conjectures.