AI Overview

Built-in AI capabilities in Nebutra β€” chat completions, embeddings, and semantic search powered by OpenAI and OpenRouter.

What's included

Nebutra ships AI features out of the box via @nebutra/agents, a thin wrapper around the Vercel AI SDK that adds tenant context, rate limiting, and usage metering automatically.

The following capabilities are available to every tenant:

FeatureDescriptionFeature Flag
Chat completionsStreaming and non-streaming LLM responsesai.chat
EmbeddingsVector representations for semantic search and RAGai.embeddings
Image analysisVision-capable model support (GPT-5)ai.chat
Model selectionPer-tenant model overrides (Enterprise)ai.model_selection

Architecture

Requests flow through the API gateway, which applies auth, rate limiting, and metering before proxying to the underlying model provider:

Client App
    β”‚
    β–Ό
POST /api/v1/ai/chat          (API Gateway β€” Hono)
    β”‚
    β”œβ”€ Auth check (API key)
    β”œβ”€ Feature flag check (ai.chat)
    β”œβ”€ Rate limit (per tenant, @nebutra/rate-limit)
    β”‚
    β–Ό
@nebutra/agents               (Vercel AI SDK v6 wrapper)
    β”‚
    β”œβ”€ OpenAI  (default)
    └─ OpenRouter  (multi-model: Claude, Llama, Mistral…)
    β”‚
    β–Ό
Response / SSE stream
    β”‚
    β–Ό
@nebutra/metering             (tokens consumed β†’ billing)
    β”‚
    β–Ό
pgvector (embeddings only)    (PostgreSQL vector storage)

Supported models

ModelContextBest for
gpt-5.5~400 k+Flagship reasoning, vision, coding
gpt-5.4-mini~400 k+Fast, cost-effective tasks
text-embedding-3-smallβ€”Embeddings (1536 dims)
text-embedding-3-largeβ€”High-accuracy embeddings (3072 dims)

Ids follow models.dev. Prefer semantic tiers (flagship / fast) via @nebutra/ai-providers when possible so defaults stay current without hand-editing strings.

ModelProviderNotes
anthropic/claude-sonnet-4.6AnthropicDefault flagship reasoning
anthropic/claude-haiku-4.5AnthropicFast / cost-efficient
openai/gpt-5.5OpenAIOpenAI flagship via OpenRouter
google/gemini-3.1-pro-previewGoogleLong-context flagship
google/gemini-3.5-flashGoogleFast multimodal

Pass the full model slug to streamText / generateText. Live catalog: models.dev / OpenRouter models.

Feature flags

AI features are disabled by default and must be enabled per tenant. The flags are checked in the API gateway before any model call is made β€” no tokens are consumed if a flag is off.

// Programmatically enable AI for a tenant
import { setFeatureFlag } from "@nebutra/preset";

await setFeatureFlag("org_123", "ai.chat", true);
await setFeatureFlag("org_123", "ai.embeddings", true);

You can also toggle flags from the dashboard under Organization β†’ Features.

Environment variables

Add these to your .env (or Vercel environment variables):

# Required: at least one provider key
OPENAI_API_KEY=""

# Optional: enables multi-model routing via OpenRouter
OPENROUTER_API_KEY=""

# Default model used when none is specified per-request
AI_DEFAULT_MODEL="gpt-5.4-mini"

# Optional: override for embedding model
AI_EMBEDDING_MODEL="text-embedding-3-small"

Never expose OPENAI_API_KEY or OPENROUTER_API_KEY to the client. All model calls must go through the API gateway.

Usage and quotas

Token consumption is automatically metered via @nebutra/metering and counts against the tenant's API quota. You can inspect current usage from code or the dashboard:

import { getMetering, COMMON_METERS } from "@nebutra/metering";

const metering = await getMetering();
const quota = await metering.getQuota("org_123", "ai_tokens");
// β†’ { limit: 5000000, used: 1243000, remaining: 3757000, percentage: 0.249 }

Quota limits by plan:

PlanAI tokens / month
FREE100,000
PRO5,000,000
ENTERPRISECustom

Quick usage example

import { streamText, generateText, embed } from "@nebutra/agents";

// Streaming chat (SSE)
const result = await streamText({
  model: "gpt-5.5",
  messages: [
    { role: "system", content: "You are a helpful assistant." },
    { role: "user", content: userMessage },
  ],
  tenantId,
});

// Non-streaming
const { text } = await generateText({
  model: "gpt-5.4-mini",
  prompt: "Summarize: " + content,
  tenantId,
});

// Embeddings
const { embedding } = await embed({
  model: "text-embedding-3-small",
  value: "text to embed",
  tenantId,
});

Chatbot

Build a chatbot with the chat completions API.

Streaming

Real-time streaming responses via SSE.

Embeddings

Semantic search and RAG with pgvector.

Metering

How AI token usage counts against quotas.

How is this guide?

Edit on GitHub

Last updated on

On this page

What's included
Architecture
Supported models
Feature flags
Environment variables
Usage and quotas
Quick usage example
Related