Inference

Latest: StepFun Ships Step 5 Preview API, a 600B MoE at $1/$2.70 That Scores 44 on Intelligence Index · artificialanalysis.ai, Sep 20, 2026

Thème Mots-clés suivis : inference, latency, throughput, serving

21 AI Weekly issues and 62 live stories from the last 30 days mention Inference.

82 mentions au total 4/20 numéros récents -100% vs référence pic 2/numéro suivi depuis 2026-04-18

Tendance des mentions — 20 derniers numéros

Actus en direct (62)

ALERTE StepFun Ships Step 5 Preview API, a 600B MoE at $1/$2.70 That Scores 44 on Intelligence Index

artificialanalysis.ai · 2026-09-20 · Score : 28

ALERTE PrismML Ships Ternary Bonsai 2 27B: 5.9GB Model Retains 98.2% of Qwen3.8 27B Under Apache 2.0

marktechpost.com · 2026-09-18 · Score : 30

AutoArk's Edge0 streams a 35B MoE from SSD at 20 tok/s on a 24GB Mac mini via one-token-ahead prerouter

huggingface.co · 2026-09-17 · Score : 22

Z.ai says GLM-5.3 built the inference stack that now serves it on 100,000+ Chinese chips

z.ai · 2026-09-17 · Score : 23

IEEE Spectrum: 2026 Is the Inference Hardware Turning Point as GPUs Sit Idle 50-80% of the Decode Phase

spectrum.ieee.org · 2026-09-15 · Score : 19

Paper: Orthrus 'Lossless' Speculative Decoding Only Matches Reference 45% of the Time at BF16

huggingface.co · 2026-09-14 · Score : 23

SemiAnalysis: Datacenter Inference Beats On-Device by 46% Cost/PFLOP for Robots, But Network Wall Blocks Move

semianalysis.com · 2026-09-14 · Score : 26

Netherlands' Euclyd Raises €200M+ Series A, Taps Ex-ASML CEO Wennink to Chair Nvidia Inference Rival

cnbc.com · 2026-09-14 · Score : 23

ALERTE SemiAnalysis Benchmarks Vera Rubin NVL72 at 7x Blackwell's Tokens per MW on DeepSeek V4 Pro 1.6T

semianalysis.com · 2026-09-14 · Score : 31

XPeng AI Ships X-AuT: Progressive Audio-Encoder Compression Cuts Speech LLM Layers Without Accuracy Loss

huggingface.co · 2026-09-10 · Score : 18

Survey Paper Maps Inference-Efficiency Mechanisms Across 125 Video and Audiovisual LLMs

huggingface.co · 2026-09-10 · Score : 17

CoVeR Paper: Geometry-Only Token Pruning Retains 93.5% of Multi-View 3D VLM Accuracy at 8% of Tokens

huggingface.co · 2026-09-09 · Score : 18

ALERTE OpenBMB Ships MiniCPM5-2B, a 2.5B Apache-2.0 On-Device Model Averaging 53.9 Across 34 Benchmarks

marktechpost.com · 2026-09-07 · Score : 28

OpenVDN Drops VDN-MiniMax-H3, Hybrid-Attention Video Model Renders 14.4s at 768p in 11 Seconds

huggingface.co · 2026-09-06 · Score : 24

AMD Debuts Threadripper Halo Station at IFA, Runs Trillion-Parameter Models Locally on 576GB HBM3e

theregister.com · 2026-09-04 · Score : 25

Gimlet Labs Raises $300M at $3B Led by Andreessen Horowitz, With Arm and Microsoft's M12 Joining

bloomberg.com · 2026-09-04 · Score : 23

Random Attention Paper: Uniform Random KV-Cache Eviction Matches Best Scorers, Runs 32-43% Faster in vLLM

arxiv.org · 2026-09-04 · Score : 20

Minima Paper Fully Quantizes Qwen3.8-27B to NVFP4 W4A4 Including Gated DeltaNet, Matches BF16

huggingface.co · 2026-09-03 · Score : 26

Nvidia Launches PAIR, a Free Tool That Distributes Local AI Inference Across Home GPUs

blogs.nvidia.com · 2026-09-03 · Score : 27

Equinix, Nvidia and Together AI Launch Inference Exchange for Enterprise Open-Model Serving

cnbc.com · 2026-09-02 · Score : 23

Anthropic-Google Paper: 'Declarative Attention' Lets LLMs Skip 31–52% of Cache Reads at Minimal Accuracy Loss

huggingface.co · 2026-09-02 · Score : 25

Paper: Split Qwen3-32B Into GRPO Experts, Merge via SLERP, Match Qwen3-235B on Enterprise Traffic

huggingface.co · 2026-09-02 · Score : 19

Slotstream Runs 125B Qwen3.8 Flash Next on a 48GB Mac at 12 tok/s Via SSD-Streamed Experts

github.com · 2026-09-01 · Score : 22

The Information: Apple's Enterprise-AI Gap Left It Scrambling to Meet Mac Mini and Mac Studio Cluster Demand

macrumors.com · 2026-08-31 · Score : 19

ALERTE Perplexity Ships Hybrid Compute for Mac, Auto-Routes Sensitive Prompts to Local Qwen 3.6 and Gemma E4B

engadget.com · 2026-09-01 · Score : 28

Koboldcpp v1.120 Ships DirectIO Model Loading Plus Full Qwen 3.8-Flash-Next and Ling-3.0-Flash Support

github.com · 2026-08-29 · Score : 18

vLLM v0.28.0 Ships Decode Context Parallel, Bigger Batches and Model Runner V2 Maturation

github.com · 2026-08-29 · Score : 20

ALERTE Tencent Open-Sources Hy4-Preview, a 770B MoE With 49B Active and 1M Context Under Apache 2.0

technode.com · 2026-08-28 · Score : 31

ALERTE OpenAI Details Jalapeño, Its First Broadcom-Built Inference ASIC That Beats Nvidia Rubin on Perf-per-Watt

newsletter.semianalysis.com · 2026-08-25 · Score : 34

Apple Preps New Mac mini With M5 and M6 Chips as On-Device AI Demand Strains Supply

9to5mac.com · 2026-08-24 · Score : 18

OpenAI Reinstates 5-Hour Codex and ChatGPT Work Limits for Plus Subscribers, Cites Compute Load

9to5mac.com · 2026-08-24 · Score : 17

arXiv Paper FlashPrefill V2 Debuts Block-Sparse Prefill Attention Kernel Aimed at Long-Context LLM Serving

huggingface.co · 2026-08-21 · Score : 16

OpenAI Cuts GPT-5.6 Sol API Prices Over 20% for Three Months, Now $4 In / $20 Out

openai.com · 2026-08-21 · Score : 23

ALERTE UK AI Chip Startup Fractile Raises ~$600M at $6.5B After $250M Anthropic Inference Deal

thenextweb.com · 2026-08-19 · Score : 28

Simon Willison finds Qwen 3.8 27B defaults to 'xhigh' reasoning, wastes 22K tokens on trivial prompts

simonwillison.net · 2026-08-16 · Score : 19

ALERTE OpenAI Previews Ultrafast Mode for GPT-5.6 Sol, 14x Faster on Cerebras at 750 Tokens/Sec

helpnetsecurity.com · 2026-08-14 · Score : 28

trycua Metal Shim Unlocks 11–16x Faster llama.cpp Inference in macOS VMs on Apple Silicon

github.com · 2026-08-11 · Score : 24

Redis Creator antirez Ships h3.c, a Native Metal Runtime for MiniMax H3 Video Generation on Apple Silicon

github.com · 2026-08-11 · Score : 22

KLQ Ships Training-Free Rotation Quantization That Beats Prior W4A4KV4 Methods on Llama 3.2 and Qwen 2.5

github.com · 2026-08-09 · Score : 17

ALERTE AMD Acquires Taalas, Toronto Startup That Etches AI Model Weights Directly Into Silicon

theregister.com · 2026-08-06 · Score : 33

DeepSeek Warns Users of 'Significant' Price Hike, Reversing China's AI Price War

bloomberg.com · 2026-08-06 · Score : 23

DeepGrove Releases Maple-Preview, a 20B-A1B Ternary Reasoning MoE That Hits 218 tok/s on M4 Mac Mini

huggingface.co · 2026-08-05 · Score : 22

Alibaba's OmniPack Cuts Qwen2.5-Omni-7B FLOPs to 16.7% Without Retraining, Keeps 98% of Baseline Score

huggingface.co · 2026-08-04 · Score : 19

AMD MI300X Deployment Runs DeepSeek V4-Flash at 168 tok/s Single-Stream, Ships FP8 Correctness Fixes

github.com · 2026-08-04 · Score : 20

'Swiftlet' Runs Qwen3-Next 80B in 4.3 GB of RAM on a Mac and a 35B on an iPhone

github.com · 2026-08-03 · Score : 19

d-Matrix Buys Wallaroo.ai to Bolt MLOps Onto Corsair Inference Chip Stack

prnewswire.com · 2026-08-03 · Score : 21

Artificial Analysis: DeepSeek V4-Flash Runs at 3¢/Test vs $1.86 for GPT-5.6 Sol and $3.15 for Claude Fable 5

reuters.com · 2026-08-03 · Score : 23

Zhejiang-PolyU 'Coherent Overlap' Paper Rewrites MoE Routing Analysis, Shows Co-Experts Interact Not Complement

huggingface.co · 2026-07-31 · Score : 15

OpenAI Cuts GPT-5.6 Luna 80% and Terra 20%, Adds 2.5x Faster Sol Fast Mode on the API

openai.com · 2026-07-30 · Score : 23

NVLabs Ships Sol-Attn, Training-Free Sparse Attention That Speeds Wan 2.1 and HunyuanVideo Inference 2-5x

huggingface.co · 2026-07-27 · Score : 19

Fortune: Coinbase, DoorDash and Airbnb Turn to Cheaper Chinese AI Models Amid US Cost Squeeze

fortune.com · 2026-07-26 · Score : 20

ESP32-S3 Demo Runs a 28.9M-Parameter LLM at ~9.5 Tokens/sec on an $8 Microcontroller

github.com · 2026-07-25 · Score : 18

Echo Debuts on Hacker News, Claims Fable-Level Results From Open-Weight Model Pool at One-Third the Cost

news.ycombinator.com · 2026-07-23 · Score : 17

Etched Closes $300M Series C at $10.3B Led by Sequoia, Books ~$1B in Pre-Orders for Sohu Inference Chip

techcrunch.com · 2026-07-23 · Score : 26

Google Deprecates Temperature, Top-P and Top-K in Gemini 3.6 Flash and All Future Models

ai.google.dev · 2026-07-21 · Score : 18

Marcel Røed Releases Gigatoken on GitHub — Rust Tokenizer Runs 280–1,353x Faster Than HuggingFace, up to 681x Over Tiktoken

github.com · 2026-07-21 · Score : 17

The Information: Meta's Applied AI Engineering Incubator Building Internal OpenRouter to Cut Coding-Agent Costs

theinformation.com · 2026-07-21 · Score : 22

ALERTE Poolside Releases Laguna S 2.1, a 118B-Parameter Open-Weight MoE Coding Model That Beats DeepSeek V4-Pro Max

huggingface.co · 2026-07-21 · Score : 28

Google Building Gemini-Specific 'Frozen v2' Inference Chip Targeting 6-10x Perf/Watt vs. TPU by 2028

siliconangle.com · 2026-07-20 · Score : 26

Infinity raises $15M seed to build a CUDA alternative that runs AI models on any chip

techcrunch.com · 2026-07-20 · Score : 24

Agent Harnesses Like OpenClaw Are Changing How We Build and Run AI Models

The Register · 2026-05-17 · Score : 16

HN: δ-mem Injects Compact Delta-Rule Memory Into Frozen LLMs, Achieving 1.31× Lift on MemoryAgentBench With No Full-Context Retraining — 200 Points

arxiv.org · 2026-05-16 · Score : 19

Numéros de la newsletter (21)

#527 Schools are choosing opposite futures for AI

2026-08-30 · 1 mentions

#521 The frontier just split into three markets

2026-08-12 · 1 mentions

#520 What a week: AI became everybody's decision

2026-08-09 · 1 mentions

#508 The Cutting Edge, Across the Board

2026-06-28 · 2 mentions

AI in Education: schools quit the AI-cheating arms race

2026-06-21 · 1 mentions

Frontier Research: circuits got cheap as consciousness got loud

2026-06-21 · 2 mentions

Deep Learning: a 744B open model just lapped the labs

2026-06-21 · 5 mentions

Machine Learning: a 744B model ran across six states on raw WAN

2026-06-21 · 7 mentions

AI in Business: the agents you sell now run up your own bill

2026-06-21 · 4 mentions

AI Regulation: export-control law just became an AI kill switch

2026-06-21 · 1 mentions

Deep Learning: 4-bit pretraining just matched FP8 at scale

2026-05-26 · 14 mentions

Generative AI: DeepSeek's 75% cut rewrites the cost math

2026-05-26 · 2 mentions

Robotics: Wayve's end-to-end driver lands in Jeeps and Rams

2026-05-26 · 1 mentions

Frontier Research: LLMs reach Nature as the field bets against them

2026-05-26 · 3 mentions

Machine Learning: a reasoning model just cracked Erdős

2026-05-26 · 7 mentions

World Models: Google Ships Omni, and the LLM Monopoly Cracks

2026-05-26 · 1 mentions

AI Safety: the deepfake takedown clock starts ticking

2026-05-26 · 2 mentions

Frontier Research: reasoning is leaving the token stream

2026-05-21 · 2 mentions

Machine Learning: nobody trained bigger, everyone shrank the cache

2026-05-21 · 9 mentions

ML this week: open weights closed the gap from underneath

2026-04-25 · 7 mentions

Deep Learning: Mamba-3 halves the state. SSMs win again.

2026-04-18 · 9 mentions