Van Strien releases 380M curation BERT with open recipe

Daniel van Strien has released UltraBERT-380M Curation, a 380-million-parameter decision model for filtering training data — classifying a web page's topic, format, educational value, or a math problem's difficulty — under an Apache 2.0 licence. The model card pitches it as "A 380M-parameter decision model for data curation, with its training data and the recipe that made it, both open."

On a benchmark of 16 test units the model posts a 0.705 mean curation score, versus 0.500 for GLiClass Edge and 0.582 for d1-2B. A packed "fast path" is quoted at 531 rows per second, roughly $1.44 per million rows on an RTX PRO 6000. One training run took about an hour on the same GPU and cost around $3; three seeded runs were averaged into the final encoder.

The model card is candid about scope. It is English-only, "much weaker on general decisions than general models," and curation test sets were used during development. Van Strien tells users to "treat outputs as sorting signals, not ground truth."