TAG mentioned! at #colm2026 (by @adinawilliams.bsky.social maybe?) in context of this really cool paper on compound tokenization arxiv.org/abs/2610.055...
- CoBPE represents phrases as a base token plus reusable surface modifiers composed in embedding space, shortening sequences by about 30% versus standard BPE.
- At 780M and 1.3B parameter scales under matched training compute, CoBPE improved average downstream task performance by 1.2 points over BPE.
- Authors Yuval Reif, Guy Kaplan and Roy Schwartz frame the result as moving part of what token sequences carry into structured representations.