Bridging Compute- and Data-Optimal Pretraining

Shared on Bluesky by 2 AI experts

• Sung Kim @sungkim.bsky.social: They propose Compute-Data (CD) scaling laws, which measure the exchange rate between extra compute and fresh data. Paper: arxiv.org/abs/260… →
• A AI Firehose @ai-firehose.column.social: Harvard and MIT researchers propose Compute-Data scaling laws to optimize pretraining by evaluating derived tokens’ value. This method demon… →