MANCE claims SOTA concept erasure across 119 model tests

Concept erasure, the problem of stripping a specific attribute out of a model's internal representation without wrecking everything else, usually gets solved one domain at a time. A new arXiv paper from Matan Avitan, Yoav Goldberg, and Yanai Elazar makes a broader claim: a single method, MANCE, plus stronger variants MANCE+ and MANCE++, hitting state-of-the-art results on nonlinear concept erasure across a large sweep of settings.

The idea rests on what they call the Manifold Constraint Hypothesis. Real representations do not fill their embedding space, they concentrate on lower-dimensional manifolds, so interventions should stay on that manifold rather than editing freely across the full space. That constraint, the argument goes, is what preserves the other information a downstream system still needs after you remove the target concept. MANCE++ layers a closed-form erasure preprocessing step on top of the iterative, classifier-guided update loop.

The evaluation is what makes this a signal rather than another single-benchmark result. The paper reports 119 settings that span both text and vision, covering 13 language models, three NLP concepts, and 40 CelebA-CLIP attributes. That kind of coverage is unusual for erasure work, which has historically bounced between small BERT-era probing setups and one-off vision experiments.

The honest caveat is that state of the art on nonlinear concept erasure is a research-benchmark claim, not a deployment claim. Erasure benchmarks measure leakage against known probes; they do not guarantee that a determined adversary, or a fine-tune on downstream data, cannot recover the erased attribute. The paper also focuses on representations rather than generative outputs, which is where much of the real compliance and fairness pressure actually sits.

Still, for anyone building fairness, safety, or attribute-removal tooling on top of open representations, a unified method that behaves reasonably across both language and vision is worth tracking. Domain-specific erasure has been the norm; a modality-agnostic baseline changes what the default should be.