Multimodal

Latest: Bilibili Open-Sources Index-Translate Covering 150 Languages, Tops Out at 35B MoE Under Apache 2.0 · huggingface.co, Sep 30, 2026

Topic Tracked keywords: multimodal, vision language, omni

15 AI Weekly issues and 87 live stories from the last 30 days mention Multimodal.

37 total mentions 2/20 recent issues -100% vs. baseline peak 1/issue tracked since 2026-04-18

Mention Trend — Last 20 Issues

Live Stories (87)

Bilibili Open-Sources Index-Translate Covering 150 Languages, Tops Out at 35B MoE Under Apache 2.0

huggingface.co · 2026-09-30 · Score: 27

HF Paper: APM-Bench Benchmarks Cross-Session Persistent Memory for Egocentric Streaming Video Assistants

huggingface.co · 2026-09-30 · Score: 18

InfiniHand Paper Streams World-Space Hand Motion From Egocentric Video at 11 FPS

arxiv.org · 2026-09-28 · Score: 17

Paper: Encoder-Free Multimodal Models Predicted to Catch Encoder-Based at ~10^22 FLOPs

arxiv.org · 2026-09-28 · Score: 19

Stanford's HomeBody Wires GPT-6 Astra Directly to a Unitree G1, Skips the VLA Layer

stanford.edu · 2026-09-26 · Score: 26

Nvidia/Bar-Ilan Paper Adds KL Regularizer That Cuts Audio-Video Desynchronization From 0.804 to 0.752 in Joint Diffusion

huggingface.co · 2026-09-24 · Score: 17

RoboFollow Benchmark Shows Top VLA Policies Ace L0 but Fail Instruction Following at L1–L3

huggingface.co · 2026-09-22 · Score: 24

Qwen Team Releases OmniVChat: Native Audio-Video Dialogue Task, Bench and RL Recipe in One Drop

huggingface.co · 2026-09-21 · Score: 19

ALERT Alibaba Ships Qwen3.8-Omni-Flash, a 1M-Context Omnimodal Model With 26%+ Average Gain Over 3.5-Omni-Plus

technode.com · 2026-09-18 · Score: 29

ActionPiece VLA Tokenizer Prioritizes Physical Rank Consistency, Hits 94.8% LIBERO With Qwen3-VL-4B

arxiv.org · 2026-09-17 · Score: 24

Snap Unveils $2,195 Standalone Specs AR Glasses and Free Specs Intelligence AI Assistant

theverge.com · 2026-09-16 · Score: 23

CMU's ModAR Ships First World-Action Model That Autoregressively Denoises Depth, Point Tracks and DINO Features

huggingface.co · 2026-09-15 · Score: 20

LynnReal-Omni Paper Unifies Video Generation With 32B Multimodal DiT, Ships 27B Flash at 377ms per 540p Clip

arxiv.org · 2026-09-14 · Score: 18

'Omni-Streaming Thinking' Paper Benchmarks Streaming Video Reasoning Against GPT-6 Astra and Sol

huggingface.co · 2026-09-15 · Score: 17

PhysBrain 1.5 Paper Extends Vision-Language Models Into Physical Foundation Models

huggingface.co · 2026-09-15 · Score: 18

Agnes-AI Ships Open-Weight 33B Multimodal Agnes-3.0-Flash With 262K Context Under Apache 2.0

huggingface.co · 2026-09-11 · Score: 22

SenseNova-U1.5: 8B Native Unified MoT Beats Nano-Banana-Pro on Vision Reasoning

huggingface.co · 2026-09-10 · Score: 20

Show-Harness Paper: Off-the-Shelf VLMs Can Control Robots Zero-Shot via a Semantic Action Interface

arxiv.org · 2026-09-09 · Score: 18

CoVeR Paper: Geometry-Only Token Pruning Retains 93.5% of Multi-View 3D VLM Accuracy at 8% of Tokens

huggingface.co · 2026-09-09 · Score: 18

TANGO Ships First Whole-Body 29-DoF VLA for Humanoid Navigation, Deployed Zero-Shot on Unitree G1

huggingface.co · 2026-09-09 · Score: 22

Tencent Ships Gander, an Open Omni-Interaction Agent With 100% Turn-Taking Accuracy on Full-Duplex Bench

huggingface.co · 2026-09-09 · Score: 21

OpenAI Ships ChatGPT Images 2.5 With Sketch Input, Cuts Latency 50% Over Images 2.0

openai.com · 2026-09-08 · Score: 23

Select, Compress, Reinvest Paper: Controlled Study of Visual-Token Allocation in Long-Video MLLMs Beats Uniform Budgeting

huggingface.co · 2026-09-04 · Score: 16

LatentStream Paper Replaces Retrieval With Progressive Latent Memory for Streaming Video Understanding

huggingface.co · 2026-09-03 · Score: 18

WorldReward Paper Unifies Action-Consistency and Visual-Quality Reward for Camera-Conditioned World Models

huggingface.co · 2026-09-04 · Score: 18

Hugging Face Paper: Puffin-World Unified 3D Multimodal Model Cuts Gravity Error to 1.32°

huggingface.co · 2026-09-04 · Score: 22

DroneCATS Benchmark Finds 2B MLLMs Fly Better Than Frontier Models But Fail to Terminate Episodes

huggingface.co · 2026-09-02 · Score: 18

ALERT Fei-Fei Li's World Labs Unveils Atlas, a Multimodal World Model Generating 1-Minute 1440p Video

worldlabs.ai · 2026-09-01 · Score: 32

LightNav-0 Paper: Compact VLM Hits SOTA on 10 Navigation Sim Benchmarks With Dual-Channel Pointing

arxiv.org · 2026-08-31 · Score: 22

ALERT Runway Unveils Solaris, Its First 'Interface World Model' That Generates UIs Frame-by-Frame in Real Time

runway.com · 2026-08-31 · Score: 29

DreamX-Creator Paper Pitches Native Audio-Video Generation at 2K Resolution

huggingface.co · 2026-09-01 · Score: 18

NavMCP Scaffolds VLM + Navigation Foundation Model, Gets 78.3% Success on Unitree Go2

huggingface.co · 2026-09-01 · Score: 16

ByteDance Seed Ships Lucida, a Parse-Generate-Place VLM Pipeline for Real-to-Sim Scenes

huggingface.co · 2026-09-01 · Score: 17

ALERT DeepSeek Drops V4-Flash-Vision-Exp, a 305B MIT-Licensed Multimodal Agent Model

huggingface.co · 2026-08-31 · Score: 31

Alibaba Launches Wan3.0 Video Model, Generates 30-Second Clips From PDFs, Slides and Spreadsheets

reuters.com · 2026-08-24 · Score: 22

ALERT DeepSeek Launches V4-Flash-Vision-Exp on API, Beats Anthropic Opus 4.8 on Two Multimodal Benchmarks

siliconangle.com · 2026-08-21 · Score: 33

Apple's Camera-Equipped AirPods Confirmed in macOS Tahoe 26.7 Release Candidate Demo

macrumors.com · 2026-08-17 · Score: 22

ALERT Alibaba Ships Qwen 3.8 27B With Vision, 262K Native Context and Apache 2.0 Weights

huggingface.co · 2026-08-14 · Score: 32

Zhejiang and Yale's Sci-VBench Finds Video Generators Cluster on Realism but Split on Scientific Correctness

huggingface.co · 2026-08-10 · Score: 15

CoCoEvolve Paper: Cross-Representation Agreement Between Chart, Table and Code Yields Label-Free Training Signal

huggingface.co · 2026-08-05 · Score: 18

BridgeVLA++ Paper Adds Spatio-Temporal Memory to 3D VLA, Hits 90.5% RLBench and 96% RMBench With 10 Demos

huggingface.co · 2026-08-05 · Score: 21

Alibaba's OmniPack Cuts Qwen2.5-Omni-7B FLOPs to 16.7% Without Retraining, Keeps 98% of Baseline Score

huggingface.co · 2026-08-04 · Score: 19

Video-DeepResearch 35B-A3B Beats Claude 4.5 Sonnet by 5 Points on New VideoDR-Bench

huggingface.co · 2026-08-04 · Score: 21

PKU-Huawei DeepVoyager-VL Puts Vision Inside the Retrieval Loop for Long-Horizon Multimodal Search

huggingface.co · 2026-08-03 · Score: 20

CASIA-Led WorldExam Benchmark Finds No Video World Model Handles Camera, Action and Language Control Well

huggingface.co · 2026-08-04 · Score: 18

CASIA-Alibaba-Yale Ship UEmbed, a Decoder-Only Model That Emits Sparse and Dense Multimodal Embeddings

huggingface.co · 2026-08-03 · Score: 18

USTC-Metastone's MPIE-Bench Exposes Anatomy Failures in Multi-Person Image Editing, VLM Judges Miss 30% Gap

huggingface.co · 2026-07-30 · Score: 16

HKUST-Tsinghua LedgerMind Enforces Structured Evidence Ledgers, Cuts Multimodal Agent Fabrication 11-26pt

huggingface.co · 2026-07-31 · Score: 20

Meta and NTU's HumanCLAW Benchmark Finds Top VLMs Solve Only 16.8% of Embodied 'Find-Navigate-Sit' Tasks

huggingface.co · 2026-07-29 · Score: 22

EPFL and Apple's Modus Extends BAGEL-7B Into a Decoder-Only Any-to-Any Model Spanning 15 Modalities

huggingface.co · 2026-07-28 · Score: 26

Adobe Research and JHU Ship Wonder, a Camera-Controllable Video World Model Running 16 FPS Over Minute-Long Rollouts

huggingface.co · 2026-07-29 · Score: 20

ByteDance Ships DecoupleMix, a Convex-Optimization Framework for VLM Pretraining Data Recipes

huggingface.co · 2026-07-27 · Score: 15

Evidence Attribution Paper Shows VLMs Cite Wrong Regions Because of Coordinate Interfaces, Not Capability

huggingface.co · 2026-07-28 · Score: 20

'Show, Don't Tell' Benchmark Puts Image Generators Against VLMs on Spatial Reasoning; GPT Image 2 Solves 37% of Items GPT-5.4 Misses

huggingface.co · 2026-07-24 · Score: 17

HDR Paper: Hierarchical Denoising Delivers 76% Reasoning Gains and 54x Speedup Over Bidirectional Diffusion

huggingface.co · 2026-07-17 · Score: 20

NJU + Shanghai AI Lab Release VIABench — 761-Video, 46.9-Hour Egocentric Benchmark for Blind-Assistance MLLMs

huggingface.co · 2026-07-17 · Score: 19

VideoChat3 Released as Fully Open 4B Video MLLM With 16x Visual-Token Compression, 70.1 on Video-MME

huggingface.co · 2026-07-16 · Score: 20

OvisOCR2 Technical Report and Open-Weights Model Release — ATH-MaaS Publishes Multilingual OCR Foundation Model on Hugging Face at 42 Paper Upvotes and 134 Model Likes, Positioning It as Open Alternative to Commercial OCR APIs for Document AI Pipelines

huggingface.co · 2026-07-15 · Score: 16

Boogu-Image-0.1 Open-Source Unified Multimodal Model Released — Apache 2.0 in Four Variants (Base, Turbo, Edit, Edit-Turbo) Trained on 208.62M Images for ~$400K, Claims Parity With Nano-Banana-Pro and GPT-Image-2 on Bilingual Chinese-English Text Rendering

huggingface.co · 2026-07-16 · Score: 16

Xiaomi Robotics Publishes 'U0' — 38B Multimodal Autoregressive World Foundation Model Unifies Text-to-Image, Scene Generation, Embodied Transfer and Manipulation, Lifts π₀.₅ OOD Success From 36.9% to 63.2%

huggingface.co · 2026-07-13 · Score: 17

HF Paper 'LightMem-Ego' From ZJU Ships Streaming Multimodal Memory for Everyday Life — Hierarchical Current/Short/Long-Term Layers Route Egocentric Video and Audio Queries on Phones and AI Glasses

huggingface.co · 2026-07-13 · Score: 18

EPFL 'NEvo' Paper Hits HN: Neural-Guided Evolutionary Video Synthesis Generates AI Videos That Drive Higher Brain-Region Activation Than Handcrafted Localizer Clips

arxiv.org · 2026-07-09 · Score: 16

SenseTime Releases SenseNova-Vision Paper — Reframes All of Computer Vision as Unified Multimodal Generation, One Model Matches Task-Specialized SOTA Across Detection, OCR, Segmentation, 3D Geometry

huggingface.co · 2026-07-07 · Score: 17

HF Paper 'Nemotron-Labs-Audex-30B-A3B' — Unified Audio-Text LLM Trained on 157B Audio + 320B Text Tokens Hits SOTA on Audio/Speech Tasks Without Regressing on Text

huggingface.co · 2026-07-06 · Score: 19

HF Paper: 'Do All Visual Tokens Matter Equally?' Object-Evidence Preserving Token Merging Improves Vision-Language Retrieval at Lower Cost

huggingface.co · 2026-07-06 · Score: 16

PixWorld Paper: Pixel-Space Diffusion Unifies 3D Scene Generation and Reconstruction in a Single 1.04B Model — SOTA on WorldScore With 91.08 Camera Control and 91.39 3D Consistency

huggingface.co · 2026-07-07 · Score: 18

InternVLA-A1.5 Paper: Native-VLM Robot Policy Preserves Pretrained Semantics via Latent Foresight, Tops All Six Simulation Benchmarks

arxiv.org · 2026-07-07 · Score: 17

Alibaba Wan Team Releases Wan-Streamer v0.2 — Native Full-Duplex Audio-Video Agent Jumps to 640×368 at 25 FPS With Same ~200ms Model-Side Latency via Ulysses Context-Parallel 'Performer' Group

huggingface.co · 2026-07-07 · Score: 20

HN Front Page: 'Claude-Real-Video' Open-Source Shim Lets Any Text-Only LLM Watch a Video Frame-by-Frame — 133 Points, 41 Comments

github.com · 2026-07-02 · Score: 18

AMVL (SJTU + Ant Group): Asymmetric Mutual Variational Learning Lets MLLMs Reason in Continuous Latent Space, Gains +32 on BLINK Jigsaw and Beats Vision-R1 and DeepEyes

huggingface.co · 2026-07-01 · Score: 17

Twelve Labs Raises $100M Series B Co-Led by NEA and Naver — Nvidia-Backed Video AI Startup Signs AWS Distribution Deal

bloomberg.com · 2026-07-01 · Score: 19

HF Paper 'DataEvolver' (CSU + HKUST): Self-Evolving Multi-Agent Pipeline for Text-Rich Image Generation Data Recycles Rejected Samples, Boosts OCR-F1 by 85% on TextScenesHQ Over Static Crawl-Filter-Freeze Baselines

huggingface.co · 2026-06-30 · Score: 17

HF Paper: 'GEAR' — Guided End-to-End AutoRegression Adds Classifier-Free Guidance Directly Into Next-Token Image Generation

huggingface.co · 2026-06-30 · Score: 16

Xiaomi-GUI-0: Native Multimodal Mobile GUI Agent Hits 72% on RealMobile and 78.9% on AndroidWorld via Error-Driven Data Flywheel

huggingface.co · 2026-06-30 · Score: 18

BAAI Releases Orca World Foundation Model: Unified Next-State-Prediction Latent Space Beats Qwen3.5-4B on MVBench/TemporalBench/3DSRBench and Outperforms FLUX.2 klein on Image Prediction

huggingface.co · 2026-06-29 · Score: 21

GauntletBench: Frontier AI Agents Achieve Only 19.1% on Vision-Intensive Professional Tasks vs. 80%+ for Non-Expert Humans — Temporal, Graphical, and 3D Reasoning Named Top Gaps

huggingface.co · 2026-06-25 · Score: 21

Robust-TO: Confidence-Aware Tool Orchestration Framework Solves Video AI's 'Blind Trust Problem' — 56.4% Accuracy on Clean Inputs, Outperforms Gemini 2.5 Pro and Holds 54.3% Under Real-World Frame Corruptions

huggingface.co · 2026-06-25 · Score: 15

ViQ: Tencent HY Vision Team and Tsinghua Introduce Text-Aligned Visual Quantized Representations at Native Resolution for Unified Discrete Multimodal Modeling

huggingface.co · 2026-06-26 · Score: 17

Qwen-Image-Agent: Alibaba Proposes Agentic Framework That Closes the 'Context Gap' in Real-World Image Generation, Sets SOTA on IA-Bench, WISE-Verified, and MindBench

huggingface.co · 2026-06-25 · Score: 20

V-Zero: Answer-Label-Free On-Policy Distillation With Contrastive Evidence Gating Improves Fine-Grained Visual Reasoning at 5× Lower Training Cost Than SFT and 10× Faster Than RL Baselines

huggingface.co · 2026-06-25 · Score: 16

ShutterMuse: Fudan University and StepFun Train Unified MLLM on 130K-Sample Dataset for Real-Time Photography Composition and Pose Guidance — Introduces CaptureGuide-Bench

huggingface.co · 2026-06-25 · Score: 17

DomainShuttle: HKUST Achieves High Subject Fidelity and Cross-Domain Flexibility for Open-Domain Subject-Driven Text-to-Video Generation via Domain-MoT and DualRoPE

huggingface.co · 2026-06-24 · Score: 16

Snap's $2,195 AI AR Glasses Weigh 132 Grams, Break If You Turn Your Head, and Won't Ship for Four Months — Investors 'Actively Begging' CEO to Reconsider

404media.co · 2026-06-24 · Score: 21

Bloomberg: Apple Plans Camera-Equipped AI AirPods for Late 2027 — Stems Stream Environmental Video to Siri Rather Than Capture Photos, Alongside 20th-Anniversary iPhone and Next-Gen Foldable

macrumors.com · 2026-06-16 · Score: 25

Snap Launches $2,195 'Specs' AR Glasses at AWE 2026 — OpenAI and Gemini APIs Built In, Dual Qualcomm Processors, First Consumer Spatial Computer to Ship Before Comparable Meta Product

gizmodo.com · 2026-06-16 · Score: 26

Google Releases Android 17 With Gemini Omni, Lyria 3 Music Generation, and AudioLM Translation — Laying Groundwork for Agentic 'Gemini Intelligence' Later in 2026

techcrunch.com · 2026-06-16 · Score: 24

Zyphra Releases Zamba2-VL: Mamba2-Transformer Hybrid Vision-Language Models (1.2B–7B) With 10× Faster Time-to-First-Token Under Apache 2.0

marktechpost.com · 2026-06-12 · Score: 17

Newsletter Issues (15)

#528 What are companies building with AI? An Applied AI Deep Dive

2026-09-01 · 1 mentions

Frontier Research: circuits got cheap as consciousness got loud

2026-06-21 · 2 mentions

Generative AI: DeepSeek's 75% cut rewrites the cost math

2026-05-26 · 3 mentions

AI Regulation: Trump killed his own AI oversight order at the door

2026-05-26 · 3 mentions

AI in Marketing: Gemini eats the ad stack, agents take the keys

2026-05-26 · 1 mentions

Machine Learning: a reasoning model just cracked Erdős

2026-05-26 · 4 mentions

World Models: Google Ships Omni, and the LLM Monopoly Cracks

2026-05-26 · 6 mentions

AI Safety: the deepfake takedown clock starts ticking

2026-05-26 · 3 mentions

AI in Marketing: Google rebuilt the ad stack around one agent

2026-05-21 · 2 mentions

Machine Learning: nobody trained bigger, everyone shrank the cache

2026-05-21 · 3 mentions

World Models: Google's Genie now simulates real streets, not just dreams

2026-05-21 · 4 mentions

AI Safety: Washington floats an FDA for AI models

2026-05-21 · 1 mentions

Frontier research: the next architecture is not transformer-shaped

2026-04-25 · 1 mentions

Gen AI: $40B compute check, open-weight reset, software bloodbath

2026-04-25 · 1 mentions

AI Healthcare: Why pharma data is the real moat

2026-04-18 · 2 mentions