niklas‑stephan.de

MCP gateway options analysis: OmniRoute vs LiteLLM vs mcphub vs agentgateway (2026-11-01)

Last updated 05/11/2026 · mcp, gateway, omniroute, litellm, mcphub, agentgateway, handtrixxx

Question (2026-11-01)

Niklas re-evaluated the LiteLLM→OmniRoute switch (ai-gateway-omniroute) because OmniRoute offers no MCP gateway. Options: a standalone mcp.handtrixxx.com (MCPHub or agentgateway), or roll back to LiteLLM which claims all-in-one LLM+MCP.

Verified facts

  • OmniRoute MCP is a control-plane MCP, not a federation gateway. Source: open-sse/mcp-server/ in cheaper-inference/OmniRoute (v3.8.52 deployed). Its 110 tools are all gateway-management tools (routing, combos, cache, compression, memory, skills, pool, radar, pricing). Transports stdio/sse/streamable-http at /api/mcp/*, but the route guard tier is LOCAL_ONLY (loopback only; remote access requires an API key with manage scope) - it is not meant to serve homelab tool MCPs. There is no config to add external MCP servers and federate them.
  • LiteLLM MCP gateway exists but is experimental: litellm/proxy/_experimental/mcp_server/; serves /mcp, /{server}/mcp, /mcp/{server}, SSE, stateless+stateful streamable-HTTP sessions; MCP-as-virtual-tools in /chat/completions; per-server auth headers x-mcp-{alias}-{header}; access groups. Functional but immature (experimental namespace, rough docs, LiteLLM’s known memory/perf issues on the Python proxy).
  • MCPHub (samanhappy/mcphub, TypeScript, 2.6k stars, pushed 2026-11-01): purpose-built self-hosted MCP control plane. Dashboard, stdio+HTTP+SSE+streamable federation, per-user credentials, OAuth 2.0, smart routing, tool-result compression, MCP Apps proxy, Postgres/replica modes, one Docker image (latest includes Node/Python/uv/git; latest-full adds Rust+Playwright).
  • Agentgateway (agentgateway/agentgateway, Rust, 5.8k stars, pushed 2026-10-30, 310 open issues): “agentic proxy” = MCP gateway + its own LLM gateway + A2A + OpenAPI→MCP conversion + guardrails + JWT/OAuth/RBAC via CEL + OTel. Kubernetes-first (Gateway API controller) but has a standalone yaml mode.
  • Current MCP landscape: Outline semantic search at https://docs.niklas-stephan.de/semantic/mcp (own Traefik router, since 2026-10-31); no mcp.handtrixxx.com route exists (Alexandria was intentionally torn down, see always-on-agent-eval).

Verdict

  • The OmniRoute switch was still right. Its wins (375 providers incl. free tiers, quota-aware auto combos, compression pipeline, OIDC via authentik, working headroom chaining) are not reproducible by LiteLLM or agentgateway. LiteLLM’s MCP gateway was never the reason to keep it; it is one experimental feature. Rolling back to LiteLLM to get MCP would trade a working, tuned gateway for an experimental MCP implementation.
  • Recommended: keep OmniRoute, add a dedicated MCP gateway. For the homelab’s scale, MCPHub is the fit: MCP-only (exactly the gap), one container in /data/docker/mcp.handtrixxx.com/, dashboard, per-user credentials for Outline/Gitea/HA servers, behind Traefik + authentik forward-auth like the other services.
  • Choose agentgateway instead only if you later want one proxy for all agent traffic (LLM + MCP + A2A) with strong RBAC/observability, or want to expose OpenAPI services as MCP tools. It overlaps/conflicts with OmniRoute as an LLM gateway and is operationally heavier (Rust binary, CEL policy, OTel, 310 open issues).
  • Net effect: with MCPHub filling the MCP gap, LiteLLM has no remaining unique advantage; the LiteLLM rollback set in /data/docker/ai.handtrixxx.com/litellm-retired/ stays as an escape hatch only for LLM-gateway issues.

Open items

  • Deploy MCPHub (not yet done as of 2026-11-01): Traefik router mcp on mcp.handtrixxx.com, authentik forward-auth or MCPHub’s own user logins; decide which servers to federate (outline semantic, gitea, home assistant, ntfy?).
  • Alternative considered but rejected for now: agentgateway replacing both gateways - heavier than the homelab needs.

Follow-up (2026-11-01): full agentgateway migration instead of MCPHub?

Niklas pointed out that the actually-used OmniRoute surface is thin: routing/combos to metis llama + headroom + aux + RustySqueezer + Gemini, per-key model restrictions, OIDC dashboard login. The auto/* free-tier combos were tested 2026-11-01 and are unusable (no keyless provider works), and OmniRoute’s compression pipeline is off (Headroom does the work). So a single agentgateway (LLM + MCP in one Rust proxy) is a legitimate consolidation.

Verified against agentgateway v1.7 standalone docs:

  • LLM side: named providers (OpenAI, Gemini, Groq, Cerebras, Mistral, OpenRouter, …) plus custom provider (host/port + /v1/chat/completions) → covers metis, headroom, llama-aux, rustysqueezer unchanged. Virtual models = OmniRoute combos: failover with priority groups + health eviction; weighted and conditional (CEL) routing. LiteLLM config import tool exists.
  • MCP side: real federation (mcp.targets with stdio cmd/args or streamable-HTTP host), one multiplexed endpoint or per-server paths, per-target backend credentials, MCP authn/authz, CEL tool filtering. This is exactly the gap OmniRoute has.
  • Ops: official Docker image cr.agentgateway.dev/agentgateway:v1.7.0, single YAML config (git-manageable), SQLite for logs/analytics/budgets, Prometheus metrics on 15020, OTLP traces, OIDC authn (authentik-compatible), virtual keys with budgets. Apache-2.0; enterprise layer is Solo but standalone is complete.
  • Gaps vs OmniRoute: no 375-provider free-tier catalog and no quota-aware auto/“task-fit” combos (CEL conditional routing can be hand-built); built-in compression engines absent (irrelevant - Headroom stays chained as a custom provider); 310 open issues and Envoy-derived internals = steeper learning curve than OmniRoute’s dashboard.

Migration sketch if chosen (parallel run, then cutover): compose project /data/docker/ai.handtrixxx.com/agentgateway/ with git-tracked config.yaml; virtual model qwen3.8-flashnext = failover [headroom→metis]; ai.handtrixxx.com/v1 Traefik router repointed; new virtual keys for Zed/pi/HA; keep headroom-omni as-is. Rollback = repoint the router back to the omniroute container (it stays in the compose until the trial is clean).

Source: Wiki.js