Found first: a primary source the press has not covered yet.
A paper from Johns Hopkins finds that composing continual-learning mechanisms at the data, function, and weight levels lifts average retention from 1.2% to 34.9% when a model is fine-tuned sequentially on 100 tasks with no replay buffer and no task ID at inference. The authors call this setting "long-horizon memorization." The paper is on arXiv.
What the source says
Zheyuan Zhang, Alvin Zhang, Daniel Khashabi, and Tianmin Shu of Johns Hopkins built three 100-task query-answer memorization datasets and ran a factorial study comparing anchor types at the data, function, and weight levels, alongside merged LoRA as a low-rank allocation strategy. Naive sequential fine-tuning retains 1.2% of prior task knowledge by the end of the sequence. The best composition, all three anchor types combined with merged LoRA, reaches 34.9%, a 28-fold improvement, and places in the top 3 across all three datasets. The data anchor and merged LoRA provide the largest individual gains and interact super-additively on all three datasets.
Why it matters
Most continual learning work evaluates one mechanism at a time on short task sequences with task identifiers provided. This paper tests compositions across 100 tasks with neither, which is closer to the conditions a deployed model faces when absorbing new information over time. The super-additive interaction between the data anchor and merged LoRA suggests each targets a different source of forgetting. The composition recipe is concrete, the code is public, and a project site is at compose-cl.github.io.