Zamba2 and Nemotron-H fail new hybrid-model causality audit

Attention-mask inspection, the standard way of proving a sequence model can't peek at future tokens, caught none of 192 injected faults across eight checkpoints. A two-forward-pass audit described in a new arxiv preprint caught all 192 and localized each to the exact layer.

The paper's argument is compact. "Causality is a graph-level property, and leaks can occur via scans, aggregations, or normalization despite correct masks," the authors write. They formalize what they call prefix invariance, the property that representations at position t must not depend on future inputs, and note it is "rarely verified."

Then they name names. Static and dynamic analysis of chunked-scan code turned up the same defect in two hybrid models, Zamba2 and Nemotron-H: "an inter-chunk axis error fixed via the reference implementation." The audit itself, the authors write, "fits on one page and runs in seconds."

The abstract does not report the size of the output-distribution shift the bug causes on real prompts, or whether the fixed reference implementation has propagated into the released Zamba2 and Nemotron-H checkpoints.