FLEET Paper Adds Memory to LLM Sampling, Claims 3x Speedup and 66.2% LiveCodeBench Pass@32
Summary
FLEET represents each generation as a sparse trajectory through high-entropy states, then uses those trajectories to infer per-token utility scores that adjust logits. The paper reports the same accuracy as repeated sampling at 3x speedup, and substantially improves LiveCodeBench Pass@32 from 59.9% to 66.2% under the same budget. Authors are from Kharkiv National University of Radio Electronics.
Originally reported by huggingface.co
Read the original article →Original headline: FLEET Paper Adds Memory to LLM Sampling, Claims 3x Speedup and 66.2% LiveCodeBench Pass@32