Procedural Knowledge in Pretraining Drives Reasoning in Large Language Models
2 experts are actively discussing the implications.
“this is one of my favorite papers, similar method. It convinced me a that LLMs weren’t merely autocompleting they showed that EVEN WHEN the answer to a math problem was present in the dataset, the agent didn’t reference it, but rather looked at problems wit…”