Sakana AI's Error Diffusion trains ConvNets without backprop

There is a small but genuinely interesting result in the biologically inspired ML pile this month. Sakana AI's team put out a paper called Diffusing Blame that trains a convolutional network to 96.7% on MNIST and 61.7% on CIFAR-10 without backpropagation, and extends the same rule to reinforcement learning on Brax and Craftax. MarkTechPost has a walkthrough of the setup.

The underlying mechanism is Error Diffusion, a local rule first proposed by Kaneko in 2000. Per the reporting, each weight update depends on three signals only, and the network never transports transposed forward weights or uses random feedback matrices, which is the usual escape hatch for biologically motivated alternatives. Sakana wraps this in a dual-stream architecture where each layer is split into excitatory and inhibitory neurons living in four separate sub-matrices, all of them kept non-negative to obey Dale's principle. Only the biases are allowed to go negative. To route error to hidden units without a random feedback matrix, they use a simple modulo scheme, r(i) = i mod C, that assigns each unit to one output channel.

The classification numbers are honest about the cost. DFA, which is willing to violate Dale's principle by carrying about 2.84M negative weights, scores 97.6% MNIST and 69.1% CIFAR-10, so ED gives up roughly one to seven points to keep its biology constraints. The parameter overhead is the other tax, roughly 4x more parameters than the comparison, about 32M versus 8M. The reinforcement learning result is where ED punches above its weight: ED-PPO beats standard PPO on HalfCheetah (5494 vs 3520, p<0.001) and beats DFA-PPO on Craftax (27.0 vs 19.8).

The honest caveats are the ones the paper itself invites. This is one team, small benchmarks, and no evidence yet that Error Diffusion scales to ImageNet or transformer pretraining, so take the specifics as reported, not settled. What the reporting does not give you is the wall-clock or energy cost on real hardware, which is the whole point of the exercise for anyone thinking about neuromorphic deployment.

The forward-looking piece is who this is aimed at. As the article notes, neuromorphic and photonic hardware often encodes non-negative synaptic magnitudes physically, and ED's fixed-sign routing maps cleanly onto that substrate. That makes this less a GPU-era optimization and more a plausibility proof for the training rule you would want if the compute stack looked very different a decade from now.