Respan launches open-source Rust AI gateway built on plugins

Respan, a YC W24 startup, used its Launch HN thread to open-source a Rust AI gateway in which providers, protocols including MCP and voice, modalities such as images and video, and security features all operate as plugins. The team wrote that they had been maintaining a heavily modified version of LiteLLM in which wrapper code exceeded 100,000 lines, and that painful upgrades pushed them to rebuild around "a small core and public interfaces that teams can use to assemble the AI infrastructure their application needs."

The design borrows from Nginx. Plugins are "modular crates that you compile into your application" rather than binary downloads, and the gateway performs protocol translation between clients and providers while preserving provider-specific fields that fall outside the shared data structure. The repo is dual-licensed under MIT or Apache-2.0, and the commercial pitch is a hosted platform offering observability and unified billing on top of the open core.

Early thread commenters pushed on two things. One asked whether a plugin model trades a batteries-included baseline for supply-chain surface area, and said that "without an entire CI process," teams would struggle to adopt it. Another raised cost attribution, pointing at quirks like Groq's custom usage field. The maintainers answered that they publish GitHub release binaries and configuration templates for common cases.

It is the second gateway/serving-stack we have tracked in two days, after DeepSeek open-sourced DeepInfer on 2026-10-20.