Byteification retrofits subword LLMs to run on raw bytes

A paper in Nature describes a method the authors call "byteification," which converts existing subword-based language models into byte-level ones using 49.1B tokens of extra training. The team applies it to Olmo 3 7B, OLMo 2 1B, Qwen3 8B and Llama 3 8B, releasing the retrofitted versions as Bolmo, Bwen and Blama.

The motivation is stated plainly. Subword tokenization, the authors write, "obscures fine-grained information, which is problematic, especially for scientific data" such as code or biological sequences.

On their headline number, Bolmo 7B, derived from Olmo 3 7B, delivers "+16.5% absolute improvement in STEM tasks" over comparable byte-level models, while staying close to its subword source on general benchmarks. The procedure uses a Latent Tokenizer Language Model architecture, two stages of training with the global model's parameters frozen in stage one, and non-causal boundary prediction using one byte of future context.

The abstract gives one STEM figure and leaves the per-task breakdown, the inference-latency comparison, and the full compute cost of the 49.1B-token retrofit outside the summary. Two of the researchers we track in our Who's Who directory shared the paper the day it posted.