GitHub - FlashML-org/FreeToken

Shared on Bluesky by 2 AI experts

• Tim Kellogg @timkellogg.me: FreeToken: a serving engine specialized in MoE models optimized to run in little memory DeepSeek-V4-Flash-284B on an 8GB RTX 5090? sure,… →
• Sung Kim @sungkim.bsky.social: FreeToken: bandwidth-adaptive CPU–GPU execution + semantic-aware caching across agent turns. 3–4× faster decode, and 6–30× faster prefill t… →