Janas: Pure-C Engine Streams Qwen3-80B From NVMe at 37 tok/s

Janas, a new C-based inference engine, reports 37 tokens per second generating output from Qwen3-Next-80B, a 48 GB model, on a laptop with 32 GB of RAM. Without speculative draft models the same setup still produces 24 tok/s. llama.cpp manages 2.8.

The engine streams expert weights off NVMe storage and keeps the frequently-used experts cached in memory. On the author's Intel Core Ultra 9 185H test machine, reading a prompt clocks 62 tok/s against llama.cpp's 16. The project README states that 'Janas-LLM was faster than llama.cpp on every model compared on the same laptop', 13 models in all, in Q4_K_M quantization.

Then the caveat, in the author's own words: 'Early development. The public API and the command line change without notice, and have changed several times a week.' Maurizio 'camauri' Cammalleri, the designer, puts it blunter still: 'This is an experiment, not a product.'

The repo lists no external dependencies, a Vulkan backend, and hardware support currently limited to the development machine itself. It lands in a steady run of local-inference releases arriving from solo authors.