Paper pairs VLM 'creative brain' with image gen for editable HTML/CSS
TL;DR
- A vision-language model plans and critiques the design while an image model synthesizes isolated assets that get emitted as native HTML/CSS.
- Output keeps layers decoupled and text real rather than rasterized, so designers can post-edit — the gap diffusion models leave open.
- Validation covers posters and infographics; the abstract publishes no benchmark numbers, no baseline models, and no human-rater scores.
A team of researchers from Sun Yat-sen University and collaborators posted a paper to arXiv on September 3 proposing that a vision-language model act as the "creative brain" of a design pipeline, with an image-generation model demoted to a visual asset synthesizer.
The gap they point at is specific. Diffusion models, the authors write, produce "flattened bitmaps with error-prone text, precluding layer-wise post-editing." Code-based generators can be edited but look plain.
Their fix is an "imagine first, then act" loop: the VLM plans the layout, the image model produces isolated assets, and the system emits native HTML/CSS with real text rather than rasterized pixels, then critiques and refines through visual feedback. The abstract calls one component "Agent Design Replay" and claims it "reproduces the creative reasoning trajectory of professional designers." Users can nudge elements around with a mouse.
On posters and infographics, the paper says the approach reaches "both refined aesthetics and production-grade editability." The abstract names no baseline systems, publishes no per-domain numbers, and reports no human-rater scores.
Originally reported by paper
Read the original article →Original headline: Editable Visual Design: VLM + On-Demand Image Gen Produces Layer-Separated HTML/CSS Artifacts With Production Aesthetics