Seems like a good time to share our new preprint about model openness! (with @catherinearnett.bsky.social @tylerachang.bsky.social Pamela D. Rivière, Samuel M. Taylor @camrobjones.bsky.social @seantrott.bsky.social @rplevy.bsky.social Ben Bergen, and Micah Altman): arxiv.org/a…
AI Weekly's analysis
→
- Closed model performance can vary by up to 60% across undocumented version updates, making past benchmark results irreproducible.
- The paper defines open-weight models as providing complete weights, tokenizer, full architecture, and sufficient code to run independently.
- Authors recommend researchers justify model selection and systematically document threats to inference when using LLMs in research.
Read full analysis →