AI runtimepackages/ai/llm-gateway
llm-gateway package
Provider-routing edge for LLM calls — fallback chains, per-tenant quota, response caching, usage metering wired in.
- Stability
- Stable
- Scope
- Global
- Boundary
- packages/ai/llm-gateway
llm-gateway
packages/ai/llm-gatewayAI runtime · tool registry · MCP boundary
Ready
Capability graph
ai/llm-gatewayllm-gateway
Tool policy
Stream UI
Output
Run ledger
- 1Inputpackages/ai/llm-gateway/request
- 2Planmessage reducer -> model turn
- 3Tool policytool schema · auth scope · rate limit
- 4Stream UIevents -> UI state machine
- 5Output@nebutra/agents response envelope
p50 latency
76 ms
events/sec
409/s
providers
3
eval score
79
Usagellm-gateway.ts
typescriptllm-gateway.ts
1import { llmGateway } from "@nebutra/llm-gateway";
2
3const result = await llmGateway.run({
4 tenantId: org.id,
5 // llm-gateway is part of the AI runtime — composable with other AI primitives.
6 input: payload,
7});