Vidu S2 paper claims real-time 720p avatars and stream editing

The Vidu team has posted a paper describing Vidu S2, a two-part system: Vidu S2-Avatar, described as a 'real-time interactive digital-character model,' and Vidu S2-Editing, described as a 'real-time video editing model.'

The headline number is 720p. Compared with Vidu S1, the abstract says, S2-Avatar 'supports real-time 720p video generation, generation with dynamic references that can be updated at any moment, and stronger instruction following, such as dancing.'

The editing model works on a live stream. It supports 'style rendering, clothing replacement, character replacement, and background replacement,' per the abstract. The authors also 'explore the feasibility of real-time spatial video generation' for both components.

'Experiments show that Vidu S2 outperforms all baselines,' the paper states, without naming the baselines or publishing per-metric numbers in the abstract. A playable demo is at vidu.com/vidu-stream.