APEX shares one network for streaming video, cuts memory 4.4%
TL;DR
- APEX uses a shifted text condition to estimate fake velocity inside the generator itself, removing DMD's separate fake-score network.
- Peak GPU memory drops from 102.7 to 98.2 GiB and iteration time from 35.9 to 32.6 seconds versus DMD training.
- A 70-pair human study shows +30.0% net preference on instruction following but -8.6% on motion quality.
Peak allocated GPU memory drops from 102.7 to 98.2 GiB per GPU, and mean iteration time from 35.9 to 32.6 seconds, versus the DMD training baseline. The savings come from folding two networks into one.
The paper on Hugging Face extends APEX to streaming video using condition shifting. "The original text condition selects generation, while a shifted condition selects fake estimation using the same parameters," the abstract states. Distribution Matching Distillation's separate fake-score network goes away; the generator estimates its own fake velocity through a negated text condition, sharing parameters with the generation pass.
On the 320-case StreamAV-Bench, APEX scores 3.856 on video instruction fulfillment on the Progressive track versus S2-Avatar's 3.831, and 0.641898 to 0.632606 on visual aesthetics across both tracks. A 70-pair human study on 180-second image-to-video sequences gives APEX a +30.0% net preference on instruction following — 22 wins, 47 ties, 1 loss — and -8.6% on motion quality. "Human raters favor APEX most strongly on instruction following, with +30.0% net preference," the paper reports.
The abstract publishes no ablation isolating which component of the shared-network setup drives the motion-quality slip. It is one of 64 AI video stories on our tracker in the last 90 days.
Originally reported by huggingface.co
Read the original article →Original headline: Tsinghua and Nvidia's APEX Paper Shares One Network Between Generator and Fake-Score for Streaming Video