Paper proves contracting RNNs act as tunable state observers

Recurrent networks used for state estimation can be given formal convergence guarantees by imposing a single geometric condition on their latent dynamics, according to a paper posted to arxiv on 6 November by Vincent Andrieu, Pauline Bernard, Lucas Brivadis, Laurent Praly, and Christian Wolf, researchers tied to CNRS and affiliated control labs. Framing RNNs as a time-discretization of continuous latent dynamical systems (Neural ODEs), they show the architecture 'acts as a tunable observer: by increasing a single scalar gain that scales the latent vector field, the estimation error can be driven to an arbitrarily small neighborhood of zero at an exponentially fast rate.'

On a real mobile robotic task, their provably contracting model outperformed a vanilla GRU with 'substantially more stable training.' More interesting: the authors report that unconstrained GRUs trained on the same task naturally converge to contracting dynamics, which they read as evidence that 'contraction is an emergent property of well-trained sequence models.'

Two researchers we track flagged the paper in the days after it posted.