PLatour ships open PACT framework, questions its own gains
TL;DR
- PLatour is an open-source framework that connects a frozen large-model prefill to a frozen small-model generator through trainable zero-initialized linear bridges.
- The authors report it beats open baselines and surpasses Mostik's published math and coding results using a substantially smaller sender model.
- Controlled experiments show some gains reproduce with uninformative sender signals or receiver-only training, so the paper also publishes a PACT-Check diagnostic.
A new preprint from Zhenyi Shen and co-authors, posted to arXiv on 4 November, proposes letting a large frozen model handle the prompt prefill and a small frozen model do the generation, with a trainable "lightweight, zero-initialized linear bridge" stitched between them. The authors call the framework PLatour and the technique Prefill-to-Autoregressive Cross-Model Transfer, or PACT.
On benchmarks, the paper reports that PLatour "significantly outperforms open-source baselines on generation tasks and surpasses Mostik's reported results on shared mathematics and coding benchmarks using a substantially smaller sender." The authors' own HuggingFace model cards publish configurations pairing Qwen3-235B as the sender with a 1.7B generator on GSM8K and MATH500.
The twist is that the paper argues against itself. "Controlled experiments show that the gains on some tasks can be reproduced by uninformative sender signals, self-communication, or receiver-only training," the abstract states. The trainable bridges, the authors write, "can act as parameter-efficient adaptation mechanisms without exploiting useful sender information."
To separate genuine cross-model communication from bridge-as-adapter effects, they propose a diagnostic they call PACT-Check, built to "quantify how much of a trained PACT system's gain genuinely comes from communication." The abstract publishes no per-task PACT-Check numbers.
Originally reported by paper
Read the original article →Original headline: PLatour Ships Open Framework for Prefill-From-Large, Generate-With-Small Inference—235B→1.7B Already Running