Neural Spectral Capacity metric prunes LLaMA-7B 5,900× faster

The paper's central claim is a scalar you can compute from a Transformer's architecture spec, with no training, no data, no gradients, that ranks candidate designs better than parameter count or FLOPs. The authors, Chenyu Zhu, Ruoyu Zhao and Zhichao Lu, call it Neural Spectral Capacity (NSC) and derive it from the singular-value spectrum of each weight matrix. Under standard random initialization, they argue, "the Marchenko-Pastur law renders NSC computable from the architectural specification alone."

On FlexiBERT, restricted to architectures whose parameter counts differ by less than 10 percent, the arXiv preprint reports that "NSC outperforms #Params, #FLOPs, and representative training-free proxies in ranking across seven Transformer and CNN families," reaching a Kendall τ of 0.505 while #Params collapses to 0.082. NSC-DP, the accompanying dynamic-programming solver, "discovers a Transformer-XL architecture on WikiText-103 that beats the human-designed baseline in 2 seconds."

The same solver, the abstract says, "prunes LLaMA-7B to the best 5.7B model across eight commonsense reasoning tasks without any calibration data, about 5900x faster than the strongest training-free proxy baseline."

The abstract publishes no per-task accuracy numbers on those eight tasks, and does not name the baseline proxy being outrun.