Inference

Latest: Fractile Raises $720M at $7B Valuation, Up 6x Since May, on $250M Anthropic Chip Deal · ft.com, Nov 11, 2026

Topic Tracked keywords: inference, latency, throughput, serving

22 AI Weekly issues and 74 live stories from the last 30 days mention Inference.

83 total mentions 3/20 recent issues -100% vs. baseline peak 1/issue tracked since 2026-04-18

Mention Trend — Last 20 Issues

Live Stories (74)

Fractile Raises $720M at $7B Valuation, Up 6x Since May, on $250M Anthropic Chip Deal

ft.com · 2026-11-11 · Score: 23

llama.cpp Merges FP8 KV-Cache Support, Trimming Memory Needs for Long-Context Local Inference

github.com · 2026-11-11 · Score: 19

Apple previews Harbor, a Mac app that runs local AI models and clusters Macs over Thunderbolt 5

9to5mac.com · 2026-11-10 · Score: 25

CPU-ARC Paper: Cached Agentic Reasoning Traces Match Frontier LLMs at 16-291x Lower Cost

huggingface.co · 2026-11-09 · Score: 23

DeepX Tapes Out 2nm DX-M2 Edge Chip, Claims 20-100B Param Inference Under 5W

eetimes.com · 2026-11-09 · Score: 22

llama-modchip Runs a 284B DeepSeek V4-Flash Model in 57GB of RAM on a CPU-Only 64GB Box, Hits 9-14 Tok/s

github.com · 2026-11-08 · Score: 17

Meta Chief Strategy Officer Dave Wehner Leaves to Become CFO of $17.5B AI Inference Startup Fireworks

cnbc.com · 2026-11-07 · Score: 25

AWS SageMaker Inference Adds NVIDIA MIG Partitioning, Splits One GPU Into Up to Seven Hardware-Isolated Models

aws.amazon.com · 2026-11-06 · Score: 23

Nvidia Opens Preview of NVKE, a Bare-Metal GPU-Kubernetes Management Plane With TPM-Attested Nodes

github.com · 2026-11-05 · Score: 22

Microsoft Foundry Local Lets GitHub Copilot Run Qwen3.8-27B On-Device for Agentic Coding

devblogs.microsoft.com · 2026-11-05 · Score: 26

Arxiv 'Drex' Paper Proposes Foundation Decision Models, Beats Claude on Decision Index at 183ms

huggingface.co · 2026-11-06 · Score: 22

Subconscious, MIT, Princeton Paper Reframes Speculative Decoding as a Long-Horizon Reasoning Control Lever

huggingface.co · 2026-11-04 · Score: 23

Tencent's LoCo DPO Compresses Reasoning Lengths 42-45% on Math Without Losing Accuracy

huggingface.co · 2026-11-04 · Score: 21

ALERT Arxiv 'You Don't Need Long Context' Paper Finds 3.6-7x Cost Savings From Compaction With <3.7pp Accuracy Loss

huggingface.co · 2026-11-03 · Score: 28

DeepMem Byte-Level Architecture Beats Matched Transformer With Bottleneck Refinement at 7,600x Fewer FLOPs

huggingface.co · 2026-11-04 · Score: 20

Berkeley, Princeton, Stanford and Nvidia's SpecPilot Autonomously Discovers New Speculative-Decoding Algorithms

arxiv.org · 2026-11-03 · Score: 19

ALERT Applied Compute Lands $350M at $3.25B Led by Elad Gil, With Nvidia In and $100M ARR in 18 Months

siliconangle.com · 2026-11-03 · Score: 28

Revolut Cuts AI Coding Inference Costs ~90% by Swapping Claude Code for Self-Hosted GLM-5.3 on 56 B200 GPUs

theregister.com · 2026-11-03 · Score: 27

ALERT Anthropic Opens FlashProve With EPFL and MIT, Cryptographically Verifying AI Replies Came From Promised Weights

anthropic.com · 2026-11-02 · Score: 30

AI inference startup Fal raises $1.3B led by Bessemer at a $16B valuation, up from $8B in April

bloomberg.com · 2026-11-03 · Score: 23

Anthropic Tells IPO Investors Its 15-Person Chip Design Team Is Already Using Claude

theinformation.com · 2026-11-03 · Score: 23

Wave Forcing Hits 134.8 End-to-End FPS Streaming Video on Eight H200s, 7x Over Single-GPU Baseline

huggingface.co · 2026-11-03 · Score: 22

SemiAnalysis InferenceX Benchmark Finds AMD MI355X Serves Kimi K3 at 2.3x Lower Cost Than Nvidia B200

semianalysis.com · 2026-10-31 · Score: 22

Sam Altman Calls Local Inference 'Quite Terrible,' Says Global DRAM Should Live in Data Centers Not PCs

tomshardware.com · 2026-10-30 · Score: 19

Baseten in Talks to Raise ~$2B at Up to $30B, Doubling Its $13B June Valuation

bloomberg.com · 2026-10-30 · Score: 23

AB-Attention Training-Free Method Matches Dense Attention PSNR at High Sparsity on Wan2.1-T2V-14B

huggingface.co · 2026-10-30 · Score: 19

SpecSlot Decouples Reading, Allocation and Routing in Pixel-Space Generative Transformers

huggingface.co · 2026-10-30 · Score: 19

TAO Tool-Aware Scheduler Cuts LLM Agent Latency 26.8% by Freeing GPUs During Tool Calls

huggingface.co · 2026-10-28 · Score: 19

Apple Opens 512GB RAM Mac Studio M5 Ultra Config at $22,299 for On-Device Large-Model Workloads

macrumors.com · 2026-10-28 · Score: 19

Qualcomm Debuts AI200 and AI250 Hexagon-NPU Rack-Scale Inference Accelerators With 768GB LPDDR Per Card

tomshardware.com · 2026-10-27 · Score: 23

ISTA-DASLab Ships 2-Bit DeepSeek V4.1 Flash GGUF at 100% Task Recovery on 12GB GPU

huggingface.co · 2026-10-27 · Score: 22

Calibration-Free Channel Selection Speeds MoE Decoding 1.6-2.7x at 98% of Full-Model Accuracy

huggingface.co · 2026-10-28 · Score: 25

Microsoft's ThinkBit Keeps Qwen3 Reasoning at 2 Bits, Beats EfficientQAT by 43 MATH-500 Points at 1.7B

huggingface.co · 2026-10-27 · Score: 24

Liquid AI Opens LFM3 Family of 2B-Class Multimodal Models With 256K Context and ~1GB 4-Bit Footprint

liquid.ai · 2026-10-27 · Score: 26

General Compute Lands $100M Series A at $800M to Build Non-Nvidia Inference Cloud on SambaNova, Positron, Cerebras

techcrunch.com · 2026-10-27 · Score: 26

Cloudflare: GLM-5.3 on Workers AI Leaked 155 Responses to Later Users Via SGLang KV Cache Bug

blog.cloudflare.com · 2026-10-27 · Score: 26

Solo Dev's sFNX Engine Runs Qwen3.8-Flash-Next at 382 Tokens/s on $1,800 of Used V100s

github.com · 2026-10-26 · Score: 18

OpenAI Ships GPT-6.1 Terra With Computer Use at Scale at $1 Input/$5 Output per Million Tokens

openai.com · 2026-10-26 · Score: 22

Linux Kernel Module 'gpu-unlock' Enables Pro FP/BF16 Throughput on Nvidia RTX 3000 GPUs

github.com · 2026-10-23 · Score: 17

TTR-Compressor Reframes Sequence Compression as Test-Time Regression, Matches MLP Merger With 55.6% Fewer Vision Tokens

huggingface.co · 2026-10-23 · Score: 17

OpenAI Opens Decisions API to All at $0.10 per Million Tokens, Chases TypeSafe Jev

techcrunch.com · 2026-10-22 · Score: 27

SemiAnalysis: DeepSeek Rewrote FlashMLA KV-Cache for Huawei Ascend, Sees Full Domestic Stack by 2027-28

semianalysis.com · 2026-10-21 · Score: 17

PACE Four-Bit State Quantizer Rounds Linear-Attention LLMs Once Per Chunk, Matches 8-Bit DAMP at ~2/3 Memory

huggingface.co · 2026-10-22 · Score: 22

Specter Pairs Quantized Speculation With Expert Offloading, Hits 1.88x MoE Inference Speedup on Single GPUs

huggingface.co · 2026-10-20 · Score: 20

OpenAI Adds 12-Hour Prompt Cache Option to the API for Persistent Agents at 2.5x Write Price

developers.openai.com · 2026-10-20 · Score: 22

Large Flow Language Models (Lafa) Hit 2.54x AR Throughput at 8B With 30B Tokens of Training

huggingface.co · 2026-10-21 · Score: 25

Respan Launches at YC With Open-Source Rust AI Gateway Where Providers, Protocols and Modalities Are All Plugins

news.ycombinator.com · 2026-10-20 · Score: 18

ALERT DeepSeek Open-Sources DeepInfer, Its Production Inference Framework for V4.1-Flash on Nvidia and Huawei

github.com · 2026-10-20 · Score: 34

JKU Linz Opens KronLSTM With a Kronecker Delta Rule That Cuts Long-Context Attention Interference

huggingface.co · 2026-10-20 · Score: 22

ALERT Cloudflare Launches Clairic, a 'System One' Extraction Model 3.2x Faster and 49% Cheaper Than LLMs

blog.cloudflare.com · 2026-10-19 · Score: 29

Tel Aviv's doubleAI Debuts GPU-Kernel Optimizer, Raises $220M and Acquires 1,000 Nvidia B300s

wsj.com · 2026-10-19 · Score: 23

Vercel: TypeSafe's Jev Hit 35% of Paid Teams in Two Weeks With 0.27% of Spend, Open-Weights Now 71.5% of Gateway Tokens

techcrunch.com · 2026-10-16 · Score: 20

ALERT DeepSeek Ships V4.1-Flash-1017 With Live Web Search API on 552B Agent MoE

huggingface.co · 2026-10-17 · Score: 33

GLM-5.3 Paper Shows Naive Remote KV Cache 4.4x p99 TTFT; Sharded Store Cuts It 124x Instead

huggingface.co · 2026-10-16 · Score: 19

Open-Source PS5LM Runs GGUF Models Natively on Jailbroken PS5 at 55 Tokens/Sec on Qwen 3.5 9B

github.com · 2026-10-16 · Score: 17

ALERT Reflection AI Open-Weights Beam, a 501B Apache 2.0 Model Hitting ~80% on Terminal-Bench 2.1 at 25K Tokens

venturebeat.com · 2026-10-16 · Score: 32

NUS's Velox Routes Video World Model Attention via Frustum Slabs, 4.36x Attention and 2.43x Denoising Speedup at Chunk 48

huggingface.co · 2026-10-16 · Score: 18

'Your Transformer Is Secretly Elastic' Prunes DINOv3 ViT-H by 50% at 0.8% Accuracy Loss With No Retraining

huggingface.co · 2026-10-15 · Score: 19

Tom's Hardware: Nvidia's RTX Spark Beats M5 Max MacBook Pro by ~25% on Qwen Agent Benchmarks

tomshardware.com · 2026-10-15 · Score: 23

Tsinghua Latent Compaction Hits 96.7% RULER at 32x KV Compression via Test-Time Training

huggingface.co · 2026-10-15 · Score: 23

Modal Closes $750M Series D Led by Accel at $15.75B Valuation, Crosses $1B ARR

axios.com · 2026-10-14 · Score: 23

Shanghai AI Lab FlashVSR+ Hits 4K Video Super-Res on a Single 24GB RTX 4090 With 2x DiT Speedup

huggingface.co · 2026-10-14 · Score: 20

Character.AI Bankruptcy Filing Reveals $56.1M in Jan–Aug 2026 Compute Spend Against Peak $5.3M Monthly Revenue

404media.co · 2026-10-13 · Score: 26

Liquid AI Open-Weights LFM2.5-24B-A2B, a 2B-Active MoE Reasoning Model Hitting 184 tok/s on M5 Max

huggingface.co · 2026-10-13 · Score: 25

Apple Research Unveils Minions, a Distributed GPU Cloud Pooling Idle Apple Silicon Macs for Inference

machinelearning.apple.com · 2026-10-13 · Score: 26

Janas Streams LLM Experts From NVMe, Runs 48GB Qwen3-Next-80B on a 32GB Laptop at 24 Tokens/Second

github.com · 2026-10-12 · Score: 18

ALERT Intel Formally Launches GPU D270 at OCP Summit With 480GB LPDDR5X and 1.3 PFLOPS FP8

intel.com · 2026-10-12 · Score: 30

ALERT Microsoft Unveils Maia 300 Inference Chip on TSMC N2P, Plans Multi-Gigawatt Azure Deployment in 2027

azure.microsoft.com · 2026-10-12 · Score: 35

Amazon in Talks to Buy Decart for ~$7B, Months After Anthropic Walked From $6B Due Diligence

wsj.com · 2026-10-11 · Score: 23

Nvidia to Invest in Inference-Chip Rival d-Matrix at ~$2B Valuation, Pulling It Into NVLink Fusion

theinformation.com · 2026-10-09 · Score: 23

Cloudflare Launches Clef-Omni Multimodal, Cuts Clef-Flash Pricing 58% to $0.038/M

blog.cloudflare.com · 2026-10-09 · Score: 27

Efficient Computer raises $97M Series B at $650M valuation for dataflow AI chips

reuters.com · 2026-09-29 · Score: 21

Agent Harnesses Like OpenClaw Are Changing How We Build and Run AI Models

The Register · 2026-05-17 · Score: 16

HN: δ-mem Injects Compact Delta-Rule Memory Into Frozen LLMs, Achieving 1.31× Lift on MemoryAgentBench With No Full-Context Retraining — 200 Points

arxiv.org · 2026-05-16 · Score: 19

Newsletter Issues (22)

#535 AI got too expensive, so companies are moving to cheaper models

2026-10-06 · 1 mentions

#527 Schools are choosing opposite futures for AI

2026-08-30 · 1 mentions

#521 The frontier just split into three markets

2026-08-12 · 1 mentions

#520 What a week: AI became everybody's decision

2026-08-09 · 1 mentions

#508 The Cutting Edge, Across the Board

2026-06-28 · 2 mentions

AI in Education: schools quit the AI-cheating arms race

2026-06-21 · 1 mentions

Frontier Research: circuits got cheap as consciousness got loud

2026-06-21 · 2 mentions

Deep Learning: a 744B open model just lapped the labs

2026-06-21 · 5 mentions

Machine Learning: a 744B model ran across six states on raw WAN

2026-06-21 · 7 mentions

AI in Business: the agents you sell now run up your own bill

2026-06-21 · 4 mentions

AI Regulation: export-control law just became an AI kill switch

2026-06-21 · 1 mentions

Deep Learning: 4-bit pretraining just matched FP8 at scale

2026-05-26 · 14 mentions

Generative AI: DeepSeek's 75% cut rewrites the cost math

2026-05-26 · 2 mentions

Robotics: Wayve's end-to-end driver lands in Jeeps and Rams

2026-05-26 · 1 mentions

Frontier Research: LLMs reach Nature as the field bets against them

2026-05-26 · 3 mentions

Machine Learning: a reasoning model just cracked Erdős

2026-05-26 · 7 mentions

World Models: Google Ships Omni, and the LLM Monopoly Cracks

2026-05-26 · 1 mentions

AI Safety: the deepfake takedown clock starts ticking

2026-05-26 · 2 mentions

Frontier Research: reasoning is leaving the token stream

2026-05-21 · 2 mentions

Machine Learning: nobody trained bigger, everyone shrank the cache

2026-05-21 · 9 mentions

ML this week: open weights closed the gap from underneath

2026-04-25 · 7 mentions

Deep Learning: Mamba-3 halves the state. SSMs win again.

2026-04-18 · 9 mentions