Sapient posts 1B Hierarchical Reasoning Model on Hugging Face

Sapient Intelligence has posted HRM-Text-1B on Hugging Face: a roughly one-billion-parameter language model built on the company's Hierarchical Reasoning Model architecture, released under Apache 2.0 and trained on 40 billion unique tokens of predominantly English text. Two of the researchers we follow flagged the drop the same day it appeared.

The architecture is the point. Instead of a single stack, HRM pairs two Transformer modules — one 'high-level/slow' and one 'low-level/fast' — that iterate over the same input embeddings, with H_cycles × L_cycles set to 2 × 3, a hidden size of 1,536, 16 layers per stack, and 12 attention heads. Max sequence length is 4,096 and vocabulary is 65,536. Training used a PrefixLM objective with AdamATan2 in bfloat16.

The model card is blunt about what this checkpoint is not. "This is a **pre-alignment** model checkpoint, not a chat or instruction-following assistant," it says, adding that "if you want to use HRM-Text like a chat model, you would need to perform further alignment, such as SFT and/or RL, on task-specific data." The card also documents an inference gotcha: users must pass `token_type_ids` to preserve the PrefixLM mask, because otherwise "attention falls back to **pure causal**, which does **not** match the pre-training distribution and will give noticeably worse logits."

The listed limitations are terse: English only, no code training ("weak coding performance"), and outputs "may contain inaccuracies, biases, or unsafe contents." No comparison benchmarks against other 1B models appear on the card. The accompanying paper, credited to Guan Wang and eight co-authors, is titled "HRM-Text: Efficient Pretraining Beyond Scaling."