How much does a language model forget when finetuned on new tasks? We show both model size and optimization matter and forgetting can be nearly eliminated with self-generated replay! arxiv.org/abs/2605.26097 w/Martin Marek, Dongkyu Cho, Shikai Qiu, Rumi Chunara, and Pavel Izma…
Who's Who of AI
Andrew Gordon Wilson
Why they matter
Researcher with public evidence across AI research.
- AI signals
- 2 past 30d
- Sources
- 2 distinct domains
- Discussions
- 1 past 30d
- Latest signal
- 14d ago
Machine Learning Professor
https://cims.nyu.edu/~andrewgw
What they're sharing
[2605.26097] Forgetting in Language Models: Capacity, Optimization, and Self-Generated Replay arxiv.org
arxiv.org
GitHub - shikaiqiu/requential-coding github.com
Articles & links
How far can we compress billion-parameter LLMs? We introduce requential coding, which achieves < 1-bit per param compression, and explains why scaling doesn't hit a generalization wall! arxiv.org/pdf/2607.11883 w/ Shikai Qiu, Marc Finzi, Yujia Zheng, Kun Zhang 1/🧵
Much more in the paper! This was an incredible team effort. Check out our code at github.com/shikaiqiu/re...
Their own posts
Recent commentary
Anyone want to submit a workshop proposal all about deep learning? I think this area really might take off.
Perhaps I'm an outlier, but generally the value I derive from art is not from its backstory. I love a Bach fugue not because he was suffering, content, had many children, or whatever else, but because it's an extraordinary composition. I'd feel the same about AI generated art.
Their network
In Andrew Gordon Wilson's orbit
Center = Andrew Gordon Wilson. Left = members they follow (green edges). Right = members who follow them (blue edges). Top = mutual follows (orange edges, slightly larger). Drag any node to reposition; click to open that profile.