Kling and Peking U Transfer Video Motion Across Morphologies

Kuaishou's Kling team and Peking University have a preprint out that pushes video motion transfer past its usual constraint: source and target objects no longer need to share the same skeleton. In Motion Beyond Morphology, posted to Hugging Face Papers, the authors argue that a kangaroo's leap should be able to drive a lamp, a dragonfly's wingbeat a flower's petals, and a jellyfish's pulse a parachute, and they build the training pipeline to make that work.

The design is two stages. Stage I, which they call Abstract Motion Bootstrapping, learns a set of complementary motion views (semantic kinematics, depth-aware global trajectories, dense point tracks, structural edges, and a 6-DoF axis) and uses them to synthesize cross-category video pairs that share motion but differ in subject and background. Stage II, Cross-Category Motion Internalization, then trains the model to read those dynamics straight from a raw reference video, dropping the explicit motion-extraction step at inference. The backbone is an MMDiT-based image-to-video diffusion model with a causal 3D VAE, trained on roughly 1.6M object-centric motion clips before being fine-tuned on the constructed pairs.

To evaluate this, the team introduces OpenVMT-Dataset (10K motion-equivalent pairs, split 4K Same, 4K Near, 2K Far) and OpenVMT-Bench, whose Same, Near, and Far splits stress-test progressively larger morphological gaps across 123 image-conditioned and 166 text-conditioned test cases. The paper's claim is state-of-the-art motion fidelity and target preservation on both tracks, with a 12-evaluator preference study reporting 93.0% preference on I2V and 97.3% on T2V against the strongest open baselines they tested, DisMo and FlowMotion. Both dataset and benchmark are said to release upon acceptance.

The honest caveat is that a preprint's own benchmark, scored on its authors' own metrics, is not the same as independent replication, and the qualitative comparisons with commercial systems (Seedance 2.0, Kling Motion Brush, Kling Omni3) are picked frames rather than a full sweep. The reporting also doesn't tell you what inference costs, or whether or when any of this reaches Kling's consumer product.

What is worth watching is the direction. If cross-morphology motion transfer really does generalize the way the paper's examples suggest, the practical value for anyone building product videos, advertising, or stylized animation is that a single reference clip stops being tied to the object it was originally filmed with.