NCP-ArchPreview: LUMIA Lab Trains 8.9B Latent-Space LM That Matches OLMo-3-7B on 51% of Tokens

NCP team's 'NCP-ArchPreview' technical report scales Next Concept Prediction to 8.9B parameters on 5.73T Dolma-3 tokens. It jointly predicts tokens and product-quantized discrete concepts, matches OLMo-3-7B's loss using 51.3% of training tokens, and beats OLMo-3-7B by 2.45 points macro-average and 5.99 points on GSM8K. A 17M-parameter VQ module also delivers 4.17% speculative decoding gains on DFlash2.