⚡ 128 h early
Full-bandwidth transformer
3 experts across 2 network communities independently surfaced this.
Why this matches
Policy & governance reaction
1 attributable expert contribution
· Artificial Intelligence Papers
“Full-bandwidth transformer Xi Wang, Ziyang Cai, Zheng Zhan, Harry Dong, Ying Fan, Gustavo de Rosa, Tim Pearce, John Langford https://t.co/FUDCgqgkqW [𝚌𝚜.𝙰𝙸] https://t.co/eiA0jqzYAQ” evidence ↗
3 experts
2 communities
1 sources clustered
How the network is reacting
Experts are approaching this through 2 distinct lenses.
Multiple readings
Research & technical analysis
2 expertsEvidence, methods and technical implications.
“https://arxiv.org/abs/2608.17981 Recurrent transformer that injects activations from the top layers to the bottom layers at the next step (https://arxiv.org/abs/2608.08888). Why does this work without training? — via @rosinality https://x.com/rosinality/sta…”
Policy & governance
1 expertRules, institutions and accountability.
“Full-bandwidth transformer Xi Wang, Ziyang Cai, Zheng Zhan, Harry Dong, Ying Fan, Gustavo de Rosa, Tim Pearce, John Langford https://t.co/FUDCgqgkqW [𝚌𝚜.𝙰𝙸] https://t.co/eiA0jqzYAQ”
3 experts discussed this · 5 posts
Sung Kim: At decoding time, when you feed **previous hidden state** into the input together with token embedding, and it boosts performance for free. It unlocks the ** full bandwidth ** of the transformer: -…
Sung Kim: - It exposes past information across *depth* to the current layer/position Paper: Full-bandwidth transformer ( arxiv.org/abs/2608.08888 )
SE Gyges: u like can't backprop this tho because your path length increases without limit
Open the full discussion →