Multimodal

Latest: Show-Harness Paper: Off-the-Shelf VLMs Can Control Robots Zero-Shot via a Semantic Action Interface · arxiv.org, Sep 9, 2026

Topic Tracked keywords: multimodal, vision language, omni

16 AI Weekly issues and 83 live stories from the last 30 days mention Multimodal.

39 total mentions 2/20 recent issues -100% vs. baseline peak 1/issue tracked since 2026-04-18

Mention Trend — Last 20 Issues

Live Stories (83)

Show-Harness Paper: Off-the-Shelf VLMs Can Control Robots Zero-Shot via a Semantic Action Interface

arxiv.org · 2026-09-09 · Score: 18

CoVeR Paper: Geometry-Only Token Pruning Retains 93.5% of Multi-View 3D VLM Accuracy at 8% of Tokens

huggingface.co · 2026-09-09 · Score: 18

TANGO Ships First Whole-Body 29-DoF VLA for Humanoid Navigation, Deployed Zero-Shot on Unitree G1

huggingface.co · 2026-09-09 · Score: 22

Tencent Ships Gander, an Open Omni-Interaction Agent With 100% Turn-Taking Accuracy on Full-Duplex Bench

huggingface.co · 2026-09-09 · Score: 21

OpenAI Ships ChatGPT Images 2.5 With Sketch Input, Cuts Latency 50% Over Images 2.0

openai.com · 2026-09-08 · Score: 23

Select, Compress, Reinvest Paper: Controlled Study of Visual-Token Allocation in Long-Video MLLMs Beats Uniform Budgeting

huggingface.co · 2026-09-04 · Score: 16

LatentStream Paper Replaces Retrieval With Progressive Latent Memory for Streaming Video Understanding

huggingface.co · 2026-09-03 · Score: 18

WorldReward Paper Unifies Action-Consistency and Visual-Quality Reward for Camera-Conditioned World Models

huggingface.co · 2026-09-04 · Score: 18

Hugging Face Paper: Puffin-World Unified 3D Multimodal Model Cuts Gravity Error to 1.32°

huggingface.co · 2026-09-04 · Score: 22

DroneCATS Benchmark Finds 2B MLLMs Fly Better Than Frontier Models But Fail to Terminate Episodes

huggingface.co · 2026-09-02 · Score: 18

ALERT Fei-Fei Li's World Labs Unveils Atlas, a Multimodal World Model Generating 1-Minute 1440p Video

worldlabs.ai · 2026-09-01 · Score: 32

LightNav-0 Paper: Compact VLM Hits SOTA on 10 Navigation Sim Benchmarks With Dual-Channel Pointing

arxiv.org · 2026-08-31 · Score: 22

ALERT Runway Unveils Solaris, Its First 'Interface World Model' That Generates UIs Frame-by-Frame in Real Time

runway.com · 2026-08-31 · Score: 29

DreamX-Creator Paper Pitches Native Audio-Video Generation at 2K Resolution

huggingface.co · 2026-09-01 · Score: 18

NavMCP Scaffolds VLM + Navigation Foundation Model, Gets 78.3% Success on Unitree Go2

huggingface.co · 2026-09-01 · Score: 16

ByteDance Seed Ships Lucida, a Parse-Generate-Place VLM Pipeline for Real-to-Sim Scenes

huggingface.co · 2026-09-01 · Score: 17

ALERT DeepSeek Drops V4-Flash-Vision-Exp, a 305B MIT-Licensed Multimodal Agent Model

huggingface.co · 2026-08-31 · Score: 31

Alibaba Launches Wan3.0 Video Model, Generates 30-Second Clips From PDFs, Slides and Spreadsheets

reuters.com · 2026-08-24 · Score: 22

ALERT DeepSeek Launches V4-Flash-Vision-Exp on API, Beats Anthropic Opus 4.8 on Two Multimodal Benchmarks

siliconangle.com · 2026-08-21 · Score: 33

Apple's Camera-Equipped AirPods Confirmed in macOS Tahoe 26.7 Release Candidate Demo

macrumors.com · 2026-08-17 · Score: 22

ALERT Alibaba Ships Qwen 3.8 27B With Vision, 262K Native Context and Apache 2.0 Weights

huggingface.co · 2026-08-14 · Score: 32

Zhejiang and Yale's Sci-VBench Finds Video Generators Cluster on Realism but Split on Scientific Correctness

huggingface.co · 2026-08-10 · Score: 15

CoCoEvolve Paper: Cross-Representation Agreement Between Chart, Table and Code Yields Label-Free Training Signal

huggingface.co · 2026-08-05 · Score: 18

BridgeVLA++ Paper Adds Spatio-Temporal Memory to 3D VLA, Hits 90.5% RLBench and 96% RMBench With 10 Demos

huggingface.co · 2026-08-05 · Score: 21

Alibaba's OmniPack Cuts Qwen2.5-Omni-7B FLOPs to 16.7% Without Retraining, Keeps 98% of Baseline Score

huggingface.co · 2026-08-04 · Score: 19

Video-DeepResearch 35B-A3B Beats Claude 4.5 Sonnet by 5 Points on New VideoDR-Bench

huggingface.co · 2026-08-04 · Score: 21

PKU-Huawei DeepVoyager-VL Puts Vision Inside the Retrieval Loop for Long-Horizon Multimodal Search

huggingface.co · 2026-08-03 · Score: 20

CASIA-Led WorldExam Benchmark Finds No Video World Model Handles Camera, Action and Language Control Well

huggingface.co · 2026-08-04 · Score: 18

CASIA-Alibaba-Yale Ship UEmbed, a Decoder-Only Model That Emits Sparse and Dense Multimodal Embeddings

huggingface.co · 2026-08-03 · Score: 18

USTC-Metastone's MPIE-Bench Exposes Anatomy Failures in Multi-Person Image Editing, VLM Judges Miss 30% Gap

huggingface.co · 2026-07-30 · Score: 16

HKUST-Tsinghua LedgerMind Enforces Structured Evidence Ledgers, Cuts Multimodal Agent Fabrication 11-26pt

huggingface.co · 2026-07-31 · Score: 20

Meta and NTU's HumanCLAW Benchmark Finds Top VLMs Solve Only 16.8% of Embodied 'Find-Navigate-Sit' Tasks

huggingface.co · 2026-07-29 · Score: 22

EPFL and Apple's Modus Extends BAGEL-7B Into a Decoder-Only Any-to-Any Model Spanning 15 Modalities

huggingface.co · 2026-07-28 · Score: 26

Adobe Research and JHU Ship Wonder, a Camera-Controllable Video World Model Running 16 FPS Over Minute-Long Rollouts

huggingface.co · 2026-07-29 · Score: 20

ByteDance Ships DecoupleMix, a Convex-Optimization Framework for VLM Pretraining Data Recipes

huggingface.co · 2026-07-27 · Score: 15

Evidence Attribution Paper Shows VLMs Cite Wrong Regions Because of Coordinate Interfaces, Not Capability

huggingface.co · 2026-07-28 · Score: 20

'Show, Don't Tell' Benchmark Puts Image Generators Against VLMs on Spatial Reasoning; GPT Image 2 Solves 37% of Items GPT-5.4 Misses

huggingface.co · 2026-07-24 · Score: 17

HDR Paper: Hierarchical Denoising Delivers 76% Reasoning Gains and 54x Speedup Over Bidirectional Diffusion

huggingface.co · 2026-07-17 · Score: 20

NJU + Shanghai AI Lab Release VIABench — 761-Video, 46.9-Hour Egocentric Benchmark for Blind-Assistance MLLMs

huggingface.co · 2026-07-17 · Score: 19

VideoChat3 Released as Fully Open 4B Video MLLM With 16x Visual-Token Compression, 70.1 on Video-MME

huggingface.co · 2026-07-16 · Score: 20

OvisOCR2 Technical Report and Open-Weights Model Release — ATH-MaaS Publishes Multilingual OCR Foundation Model on Hugging Face at 42 Paper Upvotes and 134 Model Likes, Positioning It as Open Alternative to Commercial OCR APIs for Document AI Pipelines

huggingface.co · 2026-07-15 · Score: 16

Boogu-Image-0.1 Open-Source Unified Multimodal Model Released — Apache 2.0 in Four Variants (Base, Turbo, Edit, Edit-Turbo) Trained on 208.62M Images for ~$400K, Claims Parity With Nano-Banana-Pro and GPT-Image-2 on Bilingual Chinese-English Text Rendering

huggingface.co · 2026-07-16 · Score: 16

Xiaomi Robotics Publishes 'U0' — 38B Multimodal Autoregressive World Foundation Model Unifies Text-to-Image, Scene Generation, Embodied Transfer and Manipulation, Lifts π₀.₅ OOD Success From 36.9% to 63.2%

huggingface.co · 2026-07-13 · Score: 17

HF Paper 'LightMem-Ego' From ZJU Ships Streaming Multimodal Memory for Everyday Life — Hierarchical Current/Short/Long-Term Layers Route Egocentric Video and Audio Queries on Phones and AI Glasses

huggingface.co · 2026-07-13 · Score: 18

EPFL 'NEvo' Paper Hits HN: Neural-Guided Evolutionary Video Synthesis Generates AI Videos That Drive Higher Brain-Region Activation Than Handcrafted Localizer Clips

arxiv.org · 2026-07-09 · Score: 16

SenseTime Releases SenseNova-Vision Paper — Reframes All of Computer Vision as Unified Multimodal Generation, One Model Matches Task-Specialized SOTA Across Detection, OCR, Segmentation, 3D Geometry

huggingface.co · 2026-07-07 · Score: 17

HF Paper 'Nemotron-Labs-Audex-30B-A3B' — Unified Audio-Text LLM Trained on 157B Audio + 320B Text Tokens Hits SOTA on Audio/Speech Tasks Without Regressing on Text

huggingface.co · 2026-07-06 · Score: 19

HF Paper: 'Do All Visual Tokens Matter Equally?' Object-Evidence Preserving Token Merging Improves Vision-Language Retrieval at Lower Cost

huggingface.co · 2026-07-06 · Score: 16

PixWorld Paper: Pixel-Space Diffusion Unifies 3D Scene Generation and Reconstruction in a Single 1.04B Model — SOTA on WorldScore With 91.08 Camera Control and 91.39 3D Consistency

huggingface.co · 2026-07-07 · Score: 18

InternVLA-A1.5 Paper: Native-VLM Robot Policy Preserves Pretrained Semantics via Latent Foresight, Tops All Six Simulation Benchmarks

arxiv.org · 2026-07-07 · Score: 17

Alibaba Wan Team Releases Wan-Streamer v0.2 — Native Full-Duplex Audio-Video Agent Jumps to 640×368 at 25 FPS With Same ~200ms Model-Side Latency via Ulysses Context-Parallel 'Performer' Group

huggingface.co · 2026-07-07 · Score: 20

HN Front Page: 'Claude-Real-Video' Open-Source Shim Lets Any Text-Only LLM Watch a Video Frame-by-Frame — 133 Points, 41 Comments

github.com · 2026-07-02 · Score: 18

AMVL (SJTU + Ant Group): Asymmetric Mutual Variational Learning Lets MLLMs Reason in Continuous Latent Space, Gains +32 on BLINK Jigsaw and Beats Vision-R1 and DeepEyes

huggingface.co · 2026-07-01 · Score: 17

Twelve Labs Raises $100M Series B Co-Led by NEA and Naver — Nvidia-Backed Video AI Startup Signs AWS Distribution Deal

bloomberg.com · 2026-07-01 · Score: 19

HF Paper 'DataEvolver' (CSU + HKUST): Self-Evolving Multi-Agent Pipeline for Text-Rich Image Generation Data Recycles Rejected Samples, Boosts OCR-F1 by 85% on TextScenesHQ Over Static Crawl-Filter-Freeze Baselines

huggingface.co · 2026-06-30 · Score: 17

HF Paper: 'GEAR' — Guided End-to-End AutoRegression Adds Classifier-Free Guidance Directly Into Next-Token Image Generation

huggingface.co · 2026-06-30 · Score: 16

Xiaomi-GUI-0: Native Multimodal Mobile GUI Agent Hits 72% on RealMobile and 78.9% on AndroidWorld via Error-Driven Data Flywheel

huggingface.co · 2026-06-30 · Score: 18

BAAI Releases Orca World Foundation Model: Unified Next-State-Prediction Latent Space Beats Qwen3.5-4B on MVBench/TemporalBench/3DSRBench and Outperforms FLUX.2 klein on Image Prediction

huggingface.co · 2026-06-29 · Score: 21

GauntletBench: Frontier AI Agents Achieve Only 19.1% on Vision-Intensive Professional Tasks vs. 80%+ for Non-Expert Humans — Temporal, Graphical, and 3D Reasoning Named Top Gaps

huggingface.co · 2026-06-25 · Score: 21

Robust-TO: Confidence-Aware Tool Orchestration Framework Solves Video AI's 'Blind Trust Problem' — 56.4% Accuracy on Clean Inputs, Outperforms Gemini 2.5 Pro and Holds 54.3% Under Real-World Frame Corruptions

huggingface.co · 2026-06-25 · Score: 15

ViQ: Tencent HY Vision Team and Tsinghua Introduce Text-Aligned Visual Quantized Representations at Native Resolution for Unified Discrete Multimodal Modeling

huggingface.co · 2026-06-26 · Score: 17

Qwen-Image-Agent: Alibaba Proposes Agentic Framework That Closes the 'Context Gap' in Real-World Image Generation, Sets SOTA on IA-Bench, WISE-Verified, and MindBench

huggingface.co · 2026-06-25 · Score: 20

V-Zero: Answer-Label-Free On-Policy Distillation With Contrastive Evidence Gating Improves Fine-Grained Visual Reasoning at 5× Lower Training Cost Than SFT and 10× Faster Than RL Baselines

huggingface.co · 2026-06-25 · Score: 16

ShutterMuse: Fudan University and StepFun Train Unified MLLM on 130K-Sample Dataset for Real-Time Photography Composition and Pose Guidance — Introduces CaptureGuide-Bench

huggingface.co · 2026-06-25 · Score: 17

DomainShuttle: HKUST Achieves High Subject Fidelity and Cross-Domain Flexibility for Open-Domain Subject-Driven Text-to-Video Generation via Domain-MoT and DualRoPE

huggingface.co · 2026-06-24 · Score: 16

Snap's $2,195 AI AR Glasses Weigh 132 Grams, Break If You Turn Your Head, and Won't Ship for Four Months — Investors 'Actively Begging' CEO to Reconsider

404media.co · 2026-06-24 · Score: 21

Bloomberg: Apple Plans Camera-Equipped AI AirPods for Late 2027 — Stems Stream Environmental Video to Siri Rather Than Capture Photos, Alongside 20th-Anniversary iPhone and Next-Gen Foldable

macrumors.com · 2026-06-16 · Score: 25

Snap Launches $2,195 'Specs' AR Glasses at AWE 2026 — OpenAI and Gemini APIs Built In, Dual Qualcomm Processors, First Consumer Spatial Computer to Ship Before Comparable Meta Product

gizmodo.com · 2026-06-16 · Score: 26

Google Releases Android 17 With Gemini Omni, Lyria 3 Music Generation, and AudioLM Translation — Laying Groundwork for Agentic 'Gemini Intelligence' Later in 2026

techcrunch.com · 2026-06-16 · Score: 24

Zyphra Releases Zamba2-VL: Mamba2-Transformer Hybrid Vision-Language Models (1.2B–7B) With 10× Faster Time-to-First-Token Under Apache 2.0

marktechpost.com · 2026-06-12 · Score: 17

Google Launches Gemini Live 3.5 Real-Time Speech-to-Speech Translation Across 70+ Languages With Intonation and Pitch Preservation

blog.google · 2026-06-09 · Score: 24

llama.cpp Merges Video Input Support — ffmpeg-Backed Frame Extraction Brings Video to CLI, API, and Web UI Across All Vision Models

github.com · 2026-06-08 · Score: 21

Google Releases Gemma 4 12B — Open-Weight Multimodal Model Runs on 16GB Laptop, First Medium-Sized Model to Natively Ingest Audio Without a Separate Encoder

blog.google · 2026-06-03 · Score: 27

H Company Releases Holo3.1 — Open VLM Family (0.8B–35B) for Computer Use Agents With Mobile, Desktop, and Web Support

huggingface.co · 2026-06-02 · Score: 21

Alibaba Qwen Team Launches Qwen3.7-Plus on Bailian Platform, Adding Vision, Deep Reasoning, and Autonomous Iteration

marktechpost.com · 2026-06-02 · Score: 21

NVIDIA LocateAnything: Open 3B Vision-Language Grounding Model Runs 10× Faster Than Qwen3-VL Using Parallel Box Decoding

research.nvidia.com · 2026-05-27 · Score: 21

TechCrunch Hands-On: Google's Android XR Display Glasses Prototype Overlays Gemini HUD With Navigation, Translation, and Live Widgets — No Ship Date Yet

techcrunch.com · 2026-05-22 · Score: 22

NVIDIA Launches Nemotron 3 Nano Omni — Open 30B Multimodal Model Tops Six Leaderboards With 9x Throughput Over Comparable Open Omni Models

blogs.nvidia.com · 2026-05-21 · Score: 22

FullFlow (arXiv:2605.20316): Lightweight Adapters Add Bidirectional Vision-Language Generation to Pretrained Text-to-Image Flow Models Without Full Retraining

arxiv.org · 2026-05-21 · Score: 15

Google Launches Gemini Omni at I/O 2026: 'Create Anything From Any Input' Multimodal Video Model Available Now to Subscribers and Free for YouTube Shorts

TechCrunch · 2026-05-19 · Score: 25

ByteDance Releases Lance: 3B Open-Source Unified Multimodal Model Handling Image and Video Understanding, Generation, and Editing in a Single Framework

huggingface.co · 2026-05-19 · Score: 24

NYU Abu Dhabi Deploys Real-Time Bidirectional AI Sign Language Translation System Handling Arabic and English

thenationalnews.com · 2026-05-19 · Score: 15

ALERT Google I/O 2026 Keynote Live: Gemini AI Updates, Android 17, and Android XR Glasses Unveiled

gizmodo.com · 2026-05-19 · Score: 29

Newsletter Issues (16)

#528 What are companies building with AI? An Applied AI Deep Dive

2026-09-01 · 1 mentions

Frontier Research: circuits got cheap as consciousness got loud

2026-06-21 · 2 mentions

Generative AI: DeepSeek's 75% cut rewrites the cost math

2026-05-26 · 3 mentions

AI Regulation: Trump killed his own AI oversight order at the door

2026-05-26 · 3 mentions

AI in Marketing: Gemini eats the ad stack, agents take the keys

2026-05-26 · 1 mentions

Machine Learning: a reasoning model just cracked Erdős

2026-05-26 · 4 mentions

World Models: Google Ships Omni, and the LLM Monopoly Cracks

2026-05-26 · 6 mentions

AI Safety: the deepfake takedown clock starts ticking

2026-05-26 · 3 mentions

AI Regulation: the White House reverses course on AI safety

2026-05-21 · 2 mentions

AI in Marketing: Google rebuilt the ad stack around one agent

2026-05-21 · 2 mentions

Machine Learning: nobody trained bigger, everyone shrank the cache

2026-05-21 · 3 mentions

World Models: Google's Genie now simulates real streets, not just dreams

2026-05-21 · 4 mentions

AI Safety: Washington floats an FDA for AI models

2026-05-21 · 1 mentions

Frontier research: the next architecture is not transformer-shaped

2026-04-25 · 1 mentions

Gen AI: $40B compute check, open-weight reset, software bloodbath

2026-04-25 · 1 mentions

AI Healthcare: Why pharma data is the real moat

2026-04-18 · 2 mentions