AI 网关与模型路由：只在治理收益超过抽象成本时加入

Recommendation

早期直接集成一个供应商；需要集中调用层时用 AI SDK 等应用层工具作为过渡。

先证明多供应商、成本路由或组织治理是持续需求，而不是一次性的担心。¹ ²

适用范围： Application and platform teams whose provider diversity, reliability, budget, credential, policy, or centralized traffic requirements justify an AI gateway

Main trade-off

路由层提升控制和弹性，也会形成新的故障点与诊断层。¹ ² ³ ⁴

选择标准

按真实约束逐项判断，不要只比较功能列表。

Activation need
Require a real shared need for provider access, routing, fallback, retries, budgets, credentials, rate limits, caching, policy, or team governance.¹ ²
Compatibility
Test the exact models, endpoints, streaming events, tools, structured output, errors, usage, headers, state, images, audio, embeddings, and provider options required.³
Reliability and policy
Define timeout, retry, fallback, idempotency, data residency, provider selection, credential isolation, logging, redaction, retention, and outage behavior.¹ ³
Economics and exit
Verify provider pass-through, markup, credit fees, BYOK terms, cache, failed attempts, logs, seats, enterprise controls, deployment cost, export, and direct-provider escape.¹ ²

适用路线

这些条件会改变默认答案；请在作出承诺前逐项验证。

Managed AI control plane

Choose Managed AI control plane

Choose Portkey when a managed gateway plus centralized routing, credentials, budgets, policy, observability, and enterprise deployment options fit.

Verify: Verify cloud, hybrid, and air-gapped boundaries, supported providers and features, pricing-data coverage, logs, retention, guardrails, budget behavior, plans, and exit.¹

Self-operated OpenAI-compatible proxy

Choose Self-operated OpenAI-compatible proxy

Choose LiteLLM when a team wants to operate an open proxy and explicitly own provider adapters, routing, keys, budgets, logs, deployment, and upgrades.

Verify: OpenAI-compatible does not mean feature-equivalent; test every required provider behavior and own high availability, storage, security, updates, policy, and enterprise edition boundaries.²

Hosted multi-model routing

Choose Hosted multi-model routing

Choose OpenRouter when hosted access to many models and providers, consolidated billing, routing, fallback, and provider policy controls are the intended boundary.

Verify: Verify model and provider identity, routing defaults, data policy, region, fallbacks, tool support, version pinning, fees, credits, BYOK, limits, failed attempts, logs, and outage behavior.³

Vercel-managed gateway

Choose Vercel-managed gateway

Choose Vercel AI Gateway when Vercel-managed model access, routing, fallback, budgets, usage, and AI SDK integration fit the application.

Verify: Keep it distinct from Vercel hosting and AI SDK; verify provider catalog, API compatibility, provider options, credits, BYOK, pricing and markup, logs, regions, limits, and exit.⁴

边界： AI Gateways own proxy and request-path routing, fallback, retry, caching, rate, budget, credentials, policy, and traffic telemetry. LLM APIs choose models, AI SDKs own in-code abstraction, and AI Observability spans gateway and non-gateway behavior and evaluation.

关键差异

把真正会改变决策的边界单独比较。

Ownership: Managed, hosted aggregation, self-operated proxy, and platform gateway routes transfer availability, updates, policy, and support differently.¹

Compatibility: Unified or OpenAI-compatible interfaces do not guarantee provider feature, event, tool, policy, or error equivalence.³

Routing and resilience: Fallbacks can improve availability while changing provider identity, data path, latency, output behavior, and cost.² ³

Commercial boundary: Pass-through inference, platform fees, credit purchase fees, BYOK, logs, seats, and enterprise controls must be evaluated separately.³