llama-modchip runs 284B DeepSeek V4-Flash on 64GB CPU box

The open-source llama-modchip project, released under Apache-2.0 by GitHub user synchopate, claims to run DeepSeek-V4-Flash, a 284.33-billion-parameter mixture-of-experts model with 13.17 billion active parameters, entirely on a 64GB CPU-only machine. The README reports peak resident memory of 57.61 to 58.9 GiB at a 256K-token context with "zero swap," down from a 76.87 GiB baseline.

The compression lever is a custom format the repo calls Fractal-64, packing DeepSeek's weights at 1.71 bits per weight using block-64 BF16-scale tensors, with routed experts dropping as low as IQ1_S at 1.5625 bpw. For a lossless configuration the project reports "1.00000" cosine similarity and "0.0000" KL-divergence against the full-precision reference, which it labels a bit-exact match on logits.

Throughput figures come from a 16-core Zen 5 CPU with no GPU: 44.4 tok/s prefill and 9.1 to 14.7 tok/s decode at 256K context, with the upper end relying on what the README calls a "Stockfish-PVS v2" drafter, a speculative-decoding scheme that borrows the chess engine's principal-variation search and a 1.64 MB Zobrist hash table. The repo also lists verified runs on Apple M4 Max and Pro with Metal and NEON, on AWS Graviton3's Neoverse-V1 cores, and on a Threadripper PRO 7985WX whose eight CCDs push the smaller Nemotron-3.5-30B-A3B model to 111.62 tok/s.

The other half of the claim is CPU kernel work. The README describes a "4-level register (ZMM AVX-512BW) + L1D (32 KB) + L2 (1 MB) + L3 (64 MB) multi-column" tiling pattern plus register-stationary MQA8 FlashAttention, and credits those patches with "+77%" prefill and "+53%" decode uplift versus the unmodified baseline. The release lands in a dense stretch of open-source AI coverage on the tracker, arriving a day after a separate report on DeepSeek's harness crossing 1.54M daily users. Every figure above is the author's; no third party has reproduced them.