> ## Documentation Index
> Fetch the complete documentation index at: https://docs.frogbot.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# AI

> Configure providers, inference, access, hooks, and HTTP routes.

The `ai` block creates an embedded `@frogbotai/gateway` instance when FrogBot starts. Local operations use AI SDK v7 against that in-process gateway, so no separate proxy service is required. To limit each user's models and monthly spend, add the [AI policy plugin](https://docs.frogbot.ai/plugins/ai-policy.md).

***

## Configure providers

Add at least one provider. Set a provider to `true` to use its SDK's standard environment credentials.

```ts theme={"theme":"tokyo-night"}
import { buildConfig } from 'frogbot'

export default buildConfig({
  // secret, db, and collections omitted
  ai: {
    providers: {
      openai: true,
      anthropic: true,
      bedrock: true
    }
  }
})
```

Built-in keys are `openai`, `anthropic`, `google`, `bedrock`, `vertex`, `groq`, `mistral`, `cohere`, `togetherai`, `fireworks`, `deepinfra`, `xai`, `perplexity`, `cerebras`, `voyage`, `replicate`, `typesafe-ai`, `openrouter`, and `vercel`. See [TypeSafe AI setup](https://docs.frogbot.ai/gateway/providers.md#typesafe-ai-evaluation) for its server-side credential and evaluation-only models.

Model IDs use `<provider>/<model>`, where `<provider>` is the exact key you used under `ai.providers`, such as `bedrock/...` or `togetherai/...`. Replicate's `apiKey` maps to its `apiToken` option. Bedrock uses bearer tokens first, then explicit SigV4 keys, then the AWS default credential chain.

Use `{ apiKey: process.env.CUSTOM_KEY }` only for an alternate credential. Explicit keys must be non-empty. Bedrock accepts `true`, `{ region }`, paired `{ accessKeyId, secretAccessKey, sessionToken? }` overrides, or a custom `credentialProvider`. In Node.js, the default chain supports AWS profiles and IAM Identity Center, web identity, ECS task roles, and EC2 instance profiles. Install the optional `@aws-sdk/credential-providers` peer dependency to use this fallback. Edge runtimes must provide an edge-compatible `credentialProvider` instead.

Vertex AI serves Gemini and Claude through one `vertex` entry, using Google Cloud Application Default Credentials (ADC) instead of an API key:

```ts theme={"theme":"tokyo-night"}
ai: {
  defaultModel: 'vertex/gemini-3.5-flash',
  providers: {
    vertex: {
      project: 'example-project',
      location: 'global',
      anthropic: { location: 'us-east5' }
    }
  }
}
```

Call Gemini as `vertex/gemini-3.5-flash` and Claude as `vertex/claude-sonnet-4-6`. `vertex: true` reads the project and location from `GOOGLE_VERTEX_PROJECT` and `GOOGLE_VERTEX_LOCATION`. Locally, ADC uses the credentials from `gcloud auth application-default login`. In deployment, it uses the attached service account or workload identity, so no key file is needed. Pass `googleAuthOptions`, such as `{ keyFilename }` or `{ credentials }`, to use another identity. ADC needs Node.js, so the `vertex` provider doesn't run on edge runtimes. Before calling Claude, enable each Claude model for the project in the Vertex AI Model Garden, accepting Anthropic's terms there, and make sure the project has quota for it in the Claude location.

Add an OpenAI-compatible endpoint with any non-built-in key. Its `models` list is required for validation, and calls use the custom key as the provider prefix.

```ts theme={"theme":"tokyo-night"}
ai: {
  providers: {
    internal: {
      type: 'openai-compatible',
      baseUrl: 'https://models.internal.example/v1',
      apiKey: process.env.INTERNAL_AI_KEY,
      headers: { 'x-deployment': 'production' },
      models: [
        {
          id: 'legal-summary-v2',
          mode: 'chat',
          name: 'Legal Summary',
          limit: { context: 128_000, output: 8_000 },
          cost: { input: 0.5, output: 1.5 },
          modalities: { input: ['text', 'pdf'], output: ['text'] },
          tool_call: true,
          structured_output: true,
          reasoningOptions: [{ type: 'effort', values: ['low', 'high'] }]
        }
      ]
    }
  }
}
```

Invoke that model as `internal/legal-summary-v2`. The chat model picker labels it `Legal Summary`, and its reasoning selector offers `Default`, `Low`, and `High`; built-in models take their names and reasoning levels from the model catalog. Built-in provider entries accept an optional `models` allowlist to restrict their catalog IDs; this is different from a custom provider's required model definitions.

A provider's `models` list decides what the app can use, while an [agent's `model.options`](https://docs.frogbot.ai/agents/overview#define-one) decides what users can pick for that agent.

### AI options

| Option | Type | Default | Description |
| - | - | - | - |
| `providers` | `Record<string, true \| BuiltInProviderEntry \| BedrockProviderEntry \| VertexProviderEntry \| CustomProviderEntry>` | Required | Providers keyed by name. At least one entry must be set. A built-in key takes `true` or [built-in provider options](#built-in-provider-options), `bedrock` takes `true` or [Bedrock provider options](#bedrock-provider-options), `vertex` takes `true` or [Vertex provider options](#vertex-provider-options), and any other key needs [custom provider options](#custom-provider-options). |
| `routers` | `Record<string, RouterConfig>` | `{}` | Aliases from a slug to one model ID. See [Router options](#router-options). |
| `defaultModel` | `ModelId` | — | Model or router slug for agents that set no `model` and no `model.default`. Must resolve to a configured provider. |
| `smallModel` | `ModelId` | A recent low-cost model from the active model's provider | Model or router slug for internal utility calls such as naming chats. Must be a configured chat model. See [Choose a utility model](#choose-a-utility-model). |
| `transcriptionModel` | `ModelId` | — | Model or router slug the chat composer's microphone uses for speech-to-text. Must be a configured transcription model. Unset: no microphone. See [Voice input](https://docs.frogbot.ai/chat/ui-chat.md#voice-input). |
| `speechModel` | `ModelId` | — | Model or router slug that reads AI replies aloud in chat, for example `mistral/voxtral-mini-tts-latest`. Must be a configured speech model. Unset: no read aloud button. See [Read aloud](https://docs.frogbot.ai/chat/ui-chat.md#read-aloud). |
| `speechVoice` | `string` | The provider's default voice | Voice name passed to `speechModel`, such as `nova` for OpenAI. |
| `hooks` | `AIHooks` | No hooks | Lifecycle hooks for every operation. See [Observe calls](#observe-calls). |
| `access` | `AIAccessConfig` | Every category allows signed-in users | Access rules per operation category. See [Control access](#control-access). |
| `deploymentId` | `string` | `FROGBOT_DEPLOYMENT_ID`, then `'local'` | Added to every AI span as the `frogbot.deployment` attribute. |
| `telemetry` | `AITelemetryConfig` | Enabled | Span registration settings. See [Trace calls](#trace-calls). |

### Built-in provider options

Pass `true` to use the provider's standard environment credentials, or an object:

| Option | Type | Default | Description |
| - | - | - | - |
| `apiKey` | `string \| undefined` | Required | Credential for the provider. Must be a non-empty string; the key is required even though its type admits `undefined`, so `process.env` values type-check. Replicate receives it as its `apiToken`. |
| `models` | `string[]` | Every catalog model for the provider | Allowlist of the provider's catalog model names, without the provider prefix. Unknown names fail startup, and calls to other models of this provider are rejected. Also limits what an agent's `model.options: '*'` includes. |

### Bedrock provider options

Pass `true` to use bearer tokens or the AWS default credential chain, or an object with `region`, explicit keys, or a `credentialProvider`:

| Option | Type | Default | Description |
| - | - | - | - |
| `region` | `string` | `AWS_REGION` | AWS region. Required when the entry sets neither explicit keys nor `credentialProvider`. |
| `accessKeyId` | `string` | — | AWS access key ID. Requires `secretAccessKey`; can't be combined with `credentialProvider`. |
| `secretAccessKey` | `string` | — | AWS secret access key. Requires `accessKeyId`. |
| `sessionToken` | `string` | — | Temporary session token for the explicit keys. |
| `credentialProvider` | `() => Promise<{ accessKeyId: string; secretAccessKey: string; sessionToken?: string }>` | — | Returns AWS credentials for each request. Use it for edge runtimes or custom credential sources; can't be combined with explicit keys. |
| `models` | `string[]` | Every Bedrock catalog model | Allowlist of Bedrock catalog model names, as for [built-in providers](#built-in-provider-options). |

### Vertex provider options

Pass `true` to read the project and location from the environment, or an object with both:

| Option | Type | Default | Description |
| - | - | - | - |
| `project` | `string` | Required | Google Cloud project ID. Must be non-empty. |
| `location` | `string` | Required | Vertex AI location, such as `global` or `us-central1`. Must be non-empty. |
| `googleAuthOptions` | `GoogleAuthOptions` | Application Default Credentials | Passed unchanged to `google-auth-library`, such as `{ keyFilename }` or `{ credentials }`. |
| `anthropic` | `{ location: string }` | `location` | Location for Claude models, which share the project and credentials. Set it when a Claude model isn't served from the shared location. |
| `models` | `string[]` | Every Vertex catalog model | Allowlist of Vertex catalog model names, such as `gemini-3.5-flash` or `claude-sonnet-4-6`, as for [built-in providers](#built-in-provider-options). |

An `apiKey` or any other key fails startup: Vertex API keys (express mode) don't work for Claude, and the `google` provider already takes a Gemini API key.

### Custom provider options

| Option | Type | Required / default | Description |
| - | - | - | - |
| `type` | `'openai-compatible'` | Yes | Marks the entry as a custom endpoint. Its key under `ai.providers` becomes the model ID prefix and must not be a built-in provider key. |
| `baseUrl` | `string` | Yes | Base URL of the OpenAI-compatible API, usually ending in `/v1`. Must be non-empty. |
| `apiKey` | `string` | No | Sent as `Authorization: Bearer <apiKey>`. Omit it for endpoints without authentication. |
| `headers` | `Record<string, string>` | No | Extra headers sent with every request to the endpoint. |
| `models` | `ModelConfig[]` | Yes; at least one entry | The models the app can call through this endpoint. Each entry is described in [Custom model options](#custom-model-options). |

### Custom model options

| Option | Type | Required / default | Description |
| - | - | - | - |
| `id` | `string` | Yes | Model ID sent to the endpoint. Call it as `<provider key>/<id>`, such as `internal/legal-summary-v2`. Must be non-empty. |
| `mode` | `'chat' \| 'embedding' \| 'image_generation' \| 'audio_speech' \| 'audio_transcription' \| 'rerank' \| 'evaluate' \| 'video_generation'` | Yes | The kind of model. Only `chat` models can serve an agent, and only they are included by an agent's `model.options: '*'`. |
| `name` | `string` | No; the picker shows `id` | Display label in the chat model picker and in the agent manifest's `names`. |
| `cost` | `{ input?: number; output?: number; cache_read?: number }` | No; usage costs `0` | Prices in USD per million tokens. FrogBot uses them for `costUSD` in usage logs and in `calculateUsageCostUSD`, which the [AI policy plugin](https://docs.frogbot.ai/plugins/ai-policy.md) uses for spend. A missing `input` or `output` price counts as `0`. `cache_read` prices cached input tokens and defaults to the `input` price; cache-write tokens are priced as input. |
| `reasoningOptions` | `ModelReasoningOption[]` | No; `[{ type: 'effort', values: ['low', 'medium', 'high'] }]` when `reasoning` is `true`, otherwise no reasoning levels | Reasoning levels the endpoint accepts. Custom providers support `{ type: 'effort', values?: string[] }`: each value becomes a level in the chat selector, `values` defaults to `['low', 'medium', 'high']`, and a selected level is sent as `reasoning_effort`. The selector's `Default` level sends nothing. `{ type: 'budget_tokens', min?, max? }` and `{ type: 'toggle' }` are accepted by the type but add no levels for custom providers. Ignored when `reasoning` is `false`. |
| `limit` | `{ context?: number; input?: number; output?: number }` | No | Token limits for the model's context window, input, and output. Descriptive only: FrogBot does not enforce them or send them to the endpoint. |
| `modalities` | `{ input: ModelModality[]; output: ModelModality[] }` | No; every file type is sent | Content types the model accepts and produces, each one of `'text'`, `'image'`, `'audio'`, `'video'`, or `'pdf'`. In agent conversations, `input` decides which [chat attachments](https://docs.frogbot.ai/chat/overview.md#how-attachments-reach-the-agent) are sent: an image, audio, video, or PDF attachment of a type not listed is replaced with a short marker, and text attachments are always sent as text. The chat composer refuses those files when they are picked ([Attachments](https://docs.frogbot.ai/chat/ui-chat.md#attachments)). Without `modalities`, no attachment is replaced or refused for its type. `output` is descriptive only. Built-in models use the input types listed in the model catalog. |
| `reasoning` | `boolean` | No | Whether the model reasons. `true` without `reasoningOptions` offers Low, Medium, and High in the chat selector. `false` offers no levels, even when `reasoningOptions` is set. |
| `tool_call` | `boolean` | No | Whether the model supports tool calls. Descriptive only: FrogBot sends tools regardless. |
| `structured_output` | `boolean` | No | Whether the model supports structured output. Descriptive only: FrogBot does not check it before sending an `output` request. |
| `temperature` | `boolean` | No | Whether the model accepts a `temperature` setting. Descriptive only: FrogBot passes `temperature` through regardless. |
| `status` | `'alpha' \| 'beta' \| 'deprecated'` | No | Release status of the model. Descriptive only: FrogBot does not hide or reject models by status. |

***

## Add aliases

Routers are pure aliases from a stable slug to one model ID.

```ts theme={"theme":"tokyo-night"}
ai: {
  providers: {
    openai: true
  },
  routers: {
    fast: { model: 'openai/gpt-4o-mini' },
    careful: { model: 'openai/o3' }
  },
  defaultModel: 'fast'
}
```

An alias has no fallback chain or access rule. `hidden` exists on `RouterConfig` but is reserved for future admin integration.

### Router options

| Option | Type | Default | Description |
| - | - | - | - |
| `model` | `string` | Required | Provider-prefixed model ID the alias resolves to. Must be non-empty. |
| `hidden` | `boolean` | — | Reserved; FrogBot does not read it yet. |

Local operations, agents, `defaultModel`, and `smallModel` resolve aliases, and the chat manifest advertises the model behind a `transcriptionModel` alias; the mounted gateway HTTP routes expect provider-prefixed model IDs.

***

## Choose a utility model

FrogBot uses a small chat model for internal work such as naming new chats. By default, it chooses a recent low-cost model from the same provider as the active agent model. If no catalog candidate is available, it uses the active model.

Set `smallModel` to pin these calls to a configured model or alias:

```ts theme={"theme":"tokyo-night"}
ai: {
  providers: {
    openai: true
  },
  routers: {
    utility: { model: 'openai/gpt-5-nano' }
  },
  smallModel: 'utility' // or 'openai/gpt-5-nano'
}
```

The model, or the model behind the alias, must be a configured chat model. Utility calls use the normal operation lifecycle and appear in usage reporting.

***

## Call operations

Call methods on a `FrogBot` instance or `req.frogbot`. Text generation accepts either `prompt` or `messages`, never both. Messages are sent to the model as given; only agent turns [prepare attachments](https://docs.frogbot.ai/chat/overview.md#how-attachments-reach-the-agent) for the model.

```ts theme={"theme":"tokyo-night"}
const { text, usage, finishReason } = await req.frogbot.generateText({
  model: 'fast',
  prompt: 'Summarize the latest support request',
  req
})
```

These are the public v0 operation signatures.

```ts theme={"theme":"tokyo-night"}
generateText(opts: GenerateTextOpts)
streamText(opts: StreamTextOpts)
embed(opts: EmbedOpts)
embedMany(opts: EmbedManyOpts)
generateImage(opts: GenerateImageOpts)
generateSpeech(opts: GenerateSpeechOpts)
transcribe(opts: TranscribeOpts)
generateVideo(opts: GenerateVideoOpts)
rerank(opts: RerankOpts)
evaluate(opts: EvaluateOpts)
```

Every options type includes these shared options.

### Shared options

| Option | Type | Default | Description |
| - | - | - | - |
| `model` | `ModelId` | Required | Provider-prefixed model ID or router slug. A router slug resolves to its target model. |
| `req` | `Partial<FrogBotRequest>` | — | Request whose user is checked against [access](#control-access). Hooks receive it as `req`. |
| `overrideAccess` | `boolean` | `false` when `req` is passed, otherwise `true` | Set `true` to skip access checks for a call with `req`. Checks need a request, so a call without `req` always skips them. |

### Text generation options

`generateText` and `streamText` accept these options. Text generation uses the AI SDK v7 option names.

| Option | Type | Default | Description |
| - | - | - | - |
| `prompt` | `string` | Required unless `messages` is set | A single user prompt. Can't be combined with `messages`. |
| `messages` | `ModelMessage[]` | Required unless `prompt` is set | The conversation to continue. Can't be combined with `prompt`. |
| `instructions` | `string` | — | System instructions sent before the prompt or messages. |
| `tools` | `Tool[]` | — | FrogBot [tools](https://docs.frogbot.ai/agents/overview.md#tool-options) the model may call. Each runs with a [`ToolCtx`](https://docs.frogbot.ai/agents/overview#tool-context) whose `agent.slug` is `'direct'` and whose `runId` is new for the call. |
| `toolChoice` | `ToolChoice<ToolSet>` | `'auto'` | Whether and which tool the model must call: `'auto'`, `'none'`, `'required'`, or `{ type: 'tool', toolName }`. |
| `output` | `AIOutput` | Text | An `Output` specification for structured results. Read the parsed value from `result.output`. See [Request output](#request-output). |
| `stopWhen` | `StopCondition<ToolSet> \| StopCondition<ToolSet>[]` | `isStepCount(1)` | When to stop the tool loop. With the default, tool results do not trigger another model call. |
| `maxOutputTokens` | `number` | Provider default | Maximum tokens to generate. |
| `temperature` | `number` | Provider default | Sampling temperature. |
| `topP` | `number` | Provider default | Nucleus sampling threshold. |
| `topK` | `number` | Provider default | Samples only from the top K tokens. |
| `presencePenalty` | `number` | Provider default | Penalty for repeating content already present in the prompt. |
| `frequencyPenalty` | `number` | Provider default | Penalty for repeating the same words or phrases. |
| `seed` | `number` | — | Seed for deterministic sampling, when the provider supports it. |
| `maxRetries` | `number` | `2` | Retries for failed model calls. Set `0` to disable retries. |
| `timeout` | `number` | — | Total time limit for the call, in milliseconds. |
| `stopSequences` | `string[]` | — | Sequences that stop generation when produced. |
| `providerOptions` | `Record<string, unknown>` | — | Provider-specific options keyed by provider name, passed to the provider unchanged. |
| `abortSignal` | `AbortSignal` | — | Cancels the call. |
| `headers` | `Record<string, string>` | — | Extra HTTP headers sent to the provider. |
| `onStepEnd` | `(event: unknown) => void \| Promise<void>` | — | Called after each step with that step's result. |

`streamText` also accepts these callbacks:

| Option | Type | Default | Description |
| - | - | - | - |
| `onEnd` | `(event: unknown) => void \| Promise<void>` | — | Called once when the stream finishes, after FrogBot records usage and runs `afterOperation`. |
| `onFinish` | `(event: unknown) => void \| Promise<void>` | — | Alias of `onEnd`. Ignored when `onEnd` is set. |
| `onError` | `(event: { error: unknown }) => void \| Promise<void>` | — | Called when the stream fails, after `afterOperation` runs with the error. |
| `onAbort` | `(event: unknown) => void \| Promise<void>` | — | Called when `abortSignal` aborts the stream, after `afterOperation` runs with the abort reason. |

### Embedding options

`embed` takes a single `value`; `embedMany` takes `values`.

| Option | Type | Default | Description |
| - | - | - | - |
| `value` | `string` | Required for `embed` | Text to embed. |
| `values` | `string[]` | Required for `embedMany` | Texts to embed, returned in the same order. |
| `maxParallelCalls` | `number` | Unlimited | `embedMany` only. Maximum concurrent requests when the values are split across several provider calls. |
| `abortSignal` | `AbortSignal` | — | Cancels the call. |
| `headers` | `Record<string, string>` | — | Extra HTTP headers sent to the provider. |

### Image options

| Option | Type | Default | Description |
| - | - | - | - |
| `prompt` | `string` | Required | Description of the image to generate. |
| `n` | `number` | `1` | Number of images to generate. |
| `size` | `string` | Provider default | Image size as `'{width}x{height}'`, such as `'1024x1024'`. |
| `providerOptions` | `Record<string, unknown>` | — | Provider-specific options keyed by provider name. |
| `abortSignal` | `AbortSignal` | — | Cancels the call. |
| `headers` | `Record<string, string>` | — | Extra HTTP headers sent to the provider. |

### Speech options

| Option | Type | Default | Description |
| - | - | - | - |
| `text` | `string` | Required | Text to speak. |
| `voice` | `string` | Provider default | Voice name or ID. |
| `speed` | `number` | Provider default | Speaking speed. |
| `providerOptions` | `Record<string, unknown>` | — | Provider-specific options keyed by provider name. |
| `abortSignal` | `AbortSignal` | — | Cancels the call. |

### Transcription options

| Option | Type | Default | Description |
| - | - | - | - |
| `audio` | `Blob \| ReadableStream<Uint8Array> \| ArrayBuffer \| Uint8Array \| string \| URL` | Required | Audio to transcribe. A `File`, `Buffer`, or base64 string works too, and a `URL` is downloaded first. FrogBot reads a `Blob` or stream fully before calling the provider. The media type is detected from the bytes and falls back to `audio/wav`. |
| `language` | `string` | — | Language spoken in the audio, such as `'en'` or `'fr'`. Sent in the option the model's provider reads: `languageCode` for AssemblyAI and ElevenLabs, `languageCodes` for Google and Vertex, and `language` for the rest. A value set for that provider in `providerOptions` wins. |
| `providerOptions` | `Record<string, unknown>` | — | Provider-specific options keyed by provider name. |
| `abortSignal` | `AbortSignal` | — | Cancels the call. |

### Video options

| Option | Type | Default | Description |
| - | - | - | - |
| `prompt` | `string` | Required | Description of the video to generate. |
| `providerOptions` | `Record<string, unknown>` | — | Provider-specific options keyed by provider name. Use it for settings such as duration or resolution. |
| `abortSignal` | `AbortSignal` | — | Cancels the call. |

### Rerank options

| Option | Type | Default | Description |
| - | - | - | - |
| `query` | `string` | Required | Query to rank the documents against. |
| `documents` | `string[]` | Required | Documents to rank. |
| `topN` | `number` | Every document | Number of top-ranked documents to return. |
| `abortSignal` | `AbortSignal` | — | Cancels the call. |

Each method returns the result from its AI SDK v7 counterpart. `streamText` performs asynchronous access and hook setup, so await it before consuming `textStream` or another result stream.

***

## Evaluate state

Use `frogbot.evaluate()` to ask named boolean, choice, and score questions about shared state. Configure `ai.providers['typesafe-ai']` with a server-side key first; [TypeSafe AI setup](https://docs.frogbot.ai/gateway/providers.md#typesafe-ai-evaluation) covers the options. State, instructions, and criteria can contain structured JSON.

```ts theme={"theme":"tokyo-night"}
const result = await req.frogbot.evaluate({
  model: 'typesafe-ai/jev',
  state: { ticket: 'Customer was charged twice and requested a refund' },
  questions: {
    refunded: {
      type: 'boolean',
      instructions: 'Was a refund issued?'
    },
    team: {
      type: 'choice',
      instructions: 'Which team should handle the ticket?',
      criteria: { billing: 'Billing team', support: 'Support team' }
    },
    severity: {
      type: 'score',
      instructions: 'How urgent is this ticket?',
      criteria: ['Low', 'Medium', 'High']
    }
  },
  req
})

const refundProbability = result.answers.refunded.probability
const assignedTeam = result.answers.team.choice
const severityScore = result.answers.severity.score
```

The result retains inferred answer types for each question, along with SDK usage, warnings, rounding, response, and provider metadata. A boolean probability is an estimate, not an automatic decision. TypeSafe rounds probabilities and scores to two decimal places; `result.rounding` records that precision. When provided, confidence for choice and score questions is separate from their answer probabilities at `result.providerMetadata?.typesafe?.confidence`. Evaluation uses the AI SDK's experimental API, which may change in patch releases.

`typesafe-ai/jev` is FrogBot's shorthand for upstream `jev-latest`; `typesafe-ai/jev-latest` also works. Local router slugs can target either model, while HTTP requests use provider-prefixed IDs.

### Evaluate options

`evaluate` accepts the [shared options](#shared-options) plus:

| Option | Type | Default | Description |
| - | - | - | - |
| `state` | `string \| JSONObject \| JSONValue[]` | Required | The shared state every question is asked about. An array is one state, not a batch. |
| `questions` | `Record<string, EvaluationQuestion>` | Required | Named questions. Each key becomes a key of `result.answers`. See [Question options](#question-options). |
| `maxRetries` | `number` | `2` | Retries for transient provider failures. |
| `abortSignal` | `AbortSignal` | — | Cancels the call. |
| `headers` | `Record<string, string>` | — | Extra HTTP headers sent to the provider. |
| `providerOptions` | `Record<string, unknown>` | — | Provider-specific options keyed by provider name. |
| `telemetry` | `TelemetryOptions` | — | AI SDK telemetry settings for this call. |
| `runtimeContext` | `Record<string, unknown>` | — | Values passed through to telemetry callbacks. Treat it as read-only. |
| `onStart` | `(event) => void \| Promise<void>` | — | Called when the evaluation starts. |
| `onEnd` | `(event) => void \| Promise<void>` | — | Called when the evaluation completes. |

### Question options

| Option | Type | Default | Description |
| - | - | - | - |
| `type` | `'boolean' \| 'choice' \| 'score'` | Required | `boolean` answers with a `probability`, `choice` with one of the `criteria` keys, and `score` with an index into the `criteria` levels. |
| `instructions` | `string \| JSONObject \| JSONValue[]` | Required | The question to answer about the state. |
| `criteria` | `Record<string, string \| JSONObject \| JSONValue[] \| null>` for `choice`; `(string \| JSONObject \| JSONValue[] \| null)[]` for `score`; `{ true?, false? }` for `boolean` | Required for `choice` and `score` | `choice`: a non-empty map of option names to descriptions. `score`: at least two ordered levels, indexed from `0`. `boolean`: optional descriptions of what `true` and `false` mean. `null` means no description. |

***

## Request output

AI SDK v7 handles structured data through `generateText` and an `Output` specification. Read the validated value from `result.output`; there is no `generateObject` method.

```ts theme={"theme":"tokyo-night"}
import { Output } from 'frogbot'
import { z } from 'zod'

const result = await req.frogbot.generateText({
  model: 'fast',
  prompt: 'Extract the ticket number and priority',
  output: Output.object({
    schema: z.object({
      ticket: z.string(),
      priority: z.enum(['low', 'medium', 'high'])
    })
  }),
  req
})

const ticket = result.output
```

FrogBot also re-exports `Output.text`, `Output.array`, `Output.choice`, and `Output.json` through the same `Output` object.

***

## Control access

Configure access by operation category. Each rule receives `{ req, model }`, where `model` is the model ID or [alias](#add-aliases) slug the caller asked for. Every category defaults to `Boolean(req.user)`.

```ts theme={"theme":"tokyo-night"}
ai: {
  providers: {
    openai: true
  },
  access: {
    generate: ({ req, model }) => Boolean(req.user) && model !== 'openai/gpt-5.4',
    embed: ({ req }) => Boolean(req.user),
    transcribe: ({ req }) => Boolean(req.user),
    rerank: ({ req }) => Boolean(req.user),
    evaluate: ({ req }) => Boolean(req.user)
  }
}
```

| Option | Type | Default | Description |
| - | - | - | - |
| `generate` | `({ req, model }) => boolean \| Promise<boolean>` | `Boolean(req.user)` | Checked for `generateText`, `streamText`, `generateImage`, `generateSpeech`, and `generateVideo`, and for each agent model offered in chat and the [speech model](https://docs.frogbot.ai/chat/ui-chat.md#read-aloud) when chat decides whether to offer read aloud. |
| `embed` | `({ req, model }) => boolean \| Promise<boolean>` | `Boolean(req.user)` | Checked for `embed` and `embedMany`. |
| `transcribe` | `({ req, model }) => boolean \| Promise<boolean>` | `Boolean(req.user)` | Checked for `transcribe`, and for the [transcription model](https://docs.frogbot.ai/chat/ui-chat.md#voice-input) when chat decides whether to show a microphone. |
| `rerank` | `({ req, model }) => boolean \| Promise<boolean>` | `Boolean(req.user)` | Checked for `rerank`. |
| `evaluate` | `({ req, model }) => boolean \| Promise<boolean>` | `Boolean(req.user)` | Checked for `evaluate`. |

FrogBot runs the matching rule before every AI operation and [HTTP request](#call-over-http). A rule that returns `false` rejects the call with `AIAccessError` (status `403`). To send your own status and message, throw an error with a numeric `status` and a `code`; HTTP routes return it as `{ error: { message, type: code } }` with that status.

Chat also runs the `generate` rule for each of an agent's models when a signed-in user lists agents or picks a model. A model whose rule returns `false` or throws an error with a numeric `status` is hidden from that user, a request without a model uses the agent's default if allowed or else the first allowed model, and an agent with no allowed model is hidden and rejects their requests.

An in-process call with `req` enforces access unless `overrideAccess: true` is set. A server-side call without `req` bypasses access by default, since there is no user to check.

The [AI policy plugin](https://docs.frogbot.ai/plugins/ai-policy.md) builds on these rules to limit each user's models and monthly spend. Replace it with your own rules, such as credits, on the same seam.

***

## Observe calls

Register arrays of hooks for the five-phase lifecycle. FrogBot creates one lifecycle for each direct operation, local agent run, agent HTTP call, and mounted `/api/v1/*` request.

```ts theme={"theme":"tokyo-night"}
ai: {
  providers: {
    openai: true
  },
  hooks: {
    beforeOperation: [({ context, user }) => {
      context.authenticated = Boolean(user)
    }],
    beforeUpstream: [({ headers }) => {
      headers.set('x-application', 'support')
    }],
    afterOperation: [({ durationMs, error }) => {
      console.info({ durationMs, failed: Boolean(error) })
    }]
  }
}
```

| Option | Type | Default | Description |
| - | - | - | - |
| `beforeOperation` | `AIBeforeOperationHook[]` | `[]` | Runs once after alias resolution and access checks, before the AI SDK call. |
| `beforeUpstream` | `AIBeforeUpstreamHook[]` | `[]` | Runs before each model's `doGenerate`, `doStream`, or equivalent call. |
| `afterUpstream` | `AIAfterUpstreamHook[]` | `[]` | Runs after each successful model call. |
| `afterError` | `AIAfterErrorHook[]` | `[]` | Runs after a failed model call, with `failedPhase` and `error`. |
| `afterOperation` | `AIAfterOperationHook[]` | `[]` | Runs once when the full operation succeeds, fails, ends, or is aborted. |

All phases share `requestId`, `operation`, `startedAt`, mutable `context`, and mutable `otel`. The per-phase arguments match the [gateway hook phases](https://docs.frogbot.ai/gateway/hooks.md#phases). FrogBot adds these fields to every hook argument:

| Argument | Type | Description |
| - | - | - |
| `req` | `FrogBotRequest \| undefined` | The request passed to the operation or received by the HTTP route. |
| `user` | `FrogBotRequest['user'] \| undefined` | `req.user`, when a request is present. |
| `agent` | `{ slug: string; runId: string; chatId?: number \| string } \| undefined` | Set for agent runs. `chatId` is set when the run belongs to a persisted chat. |

FrogBot executes `beforeOperation` and `afterOperation` and does not pass them into the embedded gateway. It passes only `beforeUpstream`, `afterUpstream`, and `afterError` to `createGateway`.

Gateway HTTP handlers continue to execute upstream hooks inline. In-process model resolvers execute them around the returned model's `doGenerate`, `doStream`, or equivalent calls.

Mutate `messages`, `params`, `headers`, and `providerOptions` in place during `beforeUpstream`; `system` and `tools` are read-only. `afterUpstream`, `afterError`, and `afterOperation` isolate hook failures so later hooks still run.

Access failures happen before the lifecycle and run no hooks. A thrown `beforeOperation` stops the call before an operation lifecycle is returned, so `afterOperation` does not run for that failure.

***

## Trace calls

Telemetry is enabled by default and registers `@ai-sdk/otel` when that optional peer dependency is installed. Set a deployment ID or add span attributes without replacing FrogBot's defaults.

```ts theme={"theme":"tokyo-night"}
ai: {
  providers: {
    openai: true
  },
  deploymentId: 'support-production',
  telemetry: {
    enabled: true,
    enrichSpan: ({ operationId }) => ({
      'app.operation_id': operationId
    })
  }
}
```

`deploymentId` defaults to `FROGBOT_DEPLOYMENT_ID`, then `local`. When `@ai-sdk/otel` is absent, registration quietly does nothing.

| Option | Type | Default | Description |
| - | - | - | - |
| `enabled` | `boolean` | `true` | Set `false` to skip registering `@ai-sdk/otel`. |
| `enrichSpan` | `(args: AIEnrichSpanArgs) => Record<string, unknown> \| undefined` | — | Returns extra attributes for every AI SDK span. They are merged over FrogBot's `frogbot.deployment` attribute and stringified before they are attached. |

`enrichSpan` receives:

| Argument | Type | Description |
| - | - | - |
| `spanType` | `'operation' \| 'step' \| 'languageModel' \| 'tool' \| 'embedding' \| 'reranking' \| 'experimental_evaluation'` | The kind of span being created. |
| `operationId` | `string` | AI SDK operation name, such as `ai.generateText`. |
| `callId` | `string` | ID of the AI SDK call the span belongs to. |
| `runtimeContext` | `Record<string, unknown> \| undefined` | The call's runtime context. Agent runs set `{ agent: { slug, runId, chatId? } }`. |

***

## Call over HTTP

The standalone server mounts the embedded gateway at `/api/v1`. Canonical `/api/v1/*` requests are authenticated through `frogbot.auth`, and requests without a resolved user receive `401` before reaching the gateway.

Use `@frogbotai/sdk` in browser and application clients. AI methods use the SDK's configured base URL, headers, and `fetch`, so cookie and bearer authentication wrappers apply without separate AI credentials.

```ts theme={"theme":"tokyo-night"}
import { createFrogBotSDK } from '@frogbotai/sdk'

const sdk = createFrogBotSDK({
  baseURL: '/api',
  fetch: (input, init) => fetch(input, { ...init, credentials: 'include' })
})

const result = await sdk.ai.transcribe({
  model: 'groq/whisper-large-v3',
  file: recording
})

const response = await sdk.ai.chat({
  model: 'openai/gpt-4o-mini',
  messages: [{ role: 'user', content: 'Hello' }]
})

const audio = await sdk.ai.speech({
  model: 'mistral/voxtral-mini-tts-latest',
  input: 'Hello'
})
```

`transcribe` returns parsed transcription JSON. `chat` returns the raw `Response` so the caller can consume either JSON or an SSE stream. `speech` returns the audio as a `Blob`; leave out `voice` to use the provider's default.

Each accepted request gets one FrogBot lifecycle around the gateway handler. FrogBot runs the operation hooks, while the gateway handler runs the configured upstream hooks inline.

| Method | Path |
| - | - |
| `GET` | `/api/v1/models` |
| `GET` | `/api/v1/health` |
| `POST` | `/api/v1/chat/completions` |
| `POST` | `/api/v1/messages` |
| `POST` | `/api/v1/responses` |
| `POST` | `/api/v1/embeddings` |
| `POST` | `/api/v1/images/generations` |
| `POST` | `/api/v1/audio/speech` |
| `POST` | `/api/v1/audio/transcriptions` |
| `POST` | `/api/v1/videos/generations` |
| `POST` | `/api/v1/rerank` |
| `POST` | `/api/v1/evaluate` |

Pass credentials accepted by your configured auth collection. HTTP bodies use provider-prefixed model IDs because router aliases are resolved by FrogBot's local operation layer, not by the gateway handler. After authentication, mounted operation routes run the matching [`ai.access`](#control-access) rule with the body's `model`. A denied request receives `403` with `{ "error": { "message": "...", "type": "permission_error", "code": "permission_denied", "param": null } }`, or the status and code of an error the rule throws, in the same four-field body; on `/api/v1/messages` errors use Anthropic's `{ "type": "error", "error": { "type", "message", "param" } }` shape; `GET /api/v1/models` lists only the models the caller may use: each model is checked with the `ai.access` rule for its kind (`generate` for chat, image, speech and video models; `embed`, `transcribe`, `rerank` or `evaluate` otherwise), and a model whose rule returns `false` or throws an error with a `status` is left out. Keep the TypeSafe upstream key on the server; clients authenticate to FrogBot instead. See the [evaluation JSON contract](https://docs.frogbot.ai/gateway/overview.md#evaluate-over-http) for a request and response shape.

```bash theme={"theme":"tokyo-night"}
curl http://localhost:3000/api/v1/chat/completions \
  --request POST \
  --header "Authorization: Bearer $FROGBOT_TOKEN" \
  --header "Content-Type: application/json" \
  --data '{
    "model": "openai/gpt-4o-mini",
    "messages": [{ "role": "user", "content": "Hello" }]
  }'
```


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.