Gateway

@frogbotai/gateway is a standalone AI gateway built on the Vercel AI SDK. Point any OpenAI-compatible client at it and route to OpenAI, Anthropic, Google, Groq, Bedrock, Vertex, or 30+ other providers through one endpoint. It has zero FrogBot dependencies — run it from the CLI, or embed it as a fetch handler inside any server you already have.

FrogBot’s core framework embeds this exact package internally to power its ai config block — that’s the gateway running in-process, with router aliases and access control layered on top. See the FrogBot-mounted HTTP endpoints for those paths. This page and the rest of this section cover using the package standalone, outside of FrogBot.

Run it standalone

OPENAI_API_KEY=sk-... npx @frogbotai/gateway

The CLI auto-discovers providers from environment variables and listens on 0.0.0.0:3939 by default. If no providers are found, it exits with an error listing every env var it checked. See Configuration for --port, --config, and the init scaffolding command. Model IDs are namespaced as <provider>/<model> — the prefix tells the gateway which provider to dispatch to.

import OpenAI from 'openai'

const client = new OpenAI({
  apiKey: 'unused', // the gateway holds the upstream keys
  baseURL: 'http://localhost:3939/v1'
})

const res = await client.chat.completions.create({
  model: 'anthropic/claude-sonnet-4-5',
  messages: [{ role: 'user', content: 'Hello' }]
})

Embed it

createGateway() returns a WinterCG fetch handler — mount it in Hono, Next.js, Bun, Deno, or Cloudflare Workers.

import { serve } from '@hono/node-server'
import { createGateway } from '@frogbotai/gateway'

const gw = createGateway({
  providers: { openai: {} }
})

serve({ fetch: gw.handler, port: 3939 })

See Embedding for framework-specific examples.

Routes

Route	Compatibility	Streaming
`POST /v1/chat/completions`	OpenAI Chat Completions	Yes
`POST /v1/responses`	OpenAI Responses API	Yes
`POST /v1/messages`	Anthropic Messages API	Yes
`POST /v1/embeddings`	OpenAI Embeddings	—
`POST /v1/images/generations`	OpenAI Images	—
`POST /v1/audio/speech`	OpenAI Speech	—
`POST /v1/audio/transcriptions`	OpenAI Transcriptions	—
`POST /v1/rerank`	Reranking	—
`POST /v1/evaluate`	TypeSafe AI evaluation (JSON)	—
`POST /v1/videos/generations`	Video generation	—
`GET /v1/models`	OpenAI Models (catalog discovery)	—
`GET /health`	Liveness probe	—

Routes are also served at their bare paths (/chat/completions), so mounting the handler under any prefix keeps both the prefixed and bare paths working.

Evaluate over HTTP

With TypeSafe AI configured, send shared state and named questions as JSON to POST /v1/evaluate (also available at POST /evaluate). The standalone gateway also exposes gw.routes['/evaluate'].handler for selective mounting. A custom basePath replaces /v1 while keeping the bare path.

curl http://localhost:3939/v1/evaluate \
  --request POST \
  --header 'Content-Type: application/json' \
  --data '{
    "model": "typesafe-ai/jev",
    "state": { "ticket": "Customer requested a refund" },
    "questions": {
      "refunded": { "type": "boolean", "instructions": "Was a refund issued?" }
    }
  }'

The request can also include providerOptions. The JSON response has this shape (the fields inside usage appear only when reported):

type EvaluationResponse = {
  model: string
  answers: Record<string,
    | { type: 'boolean'; probability: number }
    | { type: 'choice'; choice: string; probabilities?: Record<string, number> }
    | { type: 'score'; score: number; probabilities?: Record<string, number> }
  >
  usage: { inputTokens?: number; outputTokens?: number }
  providerMetadata?: Record<string, unknown>
}

model is the provider-prefixed resolved ID if TypeSafe reports a different version, otherwise the requested ID. usage can be {} if the provider reports no counts. When available, providerMetadata.typesafe.confidence holds per-question confidence separately from answer probabilities. The response excludes raw upstream headers and body. Send provider-prefixed model IDs; typesafe-ai/jev resolves to TypeSafe’s jev-latest upstream. This endpoint uses the AI SDK’s experimental evaluation API. Secure standalone mounts with your own authentication, such as a beforeOperation hook; the gateway does not authenticate callers on its own. FrogBot’s /api/v1/evaluate mount authenticates callers and runs its ai.access.evaluate rule with the requested model before forwarding requests. Keep the TypeSafe API key on the server, never in the request body or client code. The gateway does not forward a caller’s Authorization header to TypeSafe.

Next steps

Providers

36+ built-in providers, plus any self-hosted OpenAI-compatible endpoint.

Configuration

gateway.config.ts, CLI flags, and allow/deny lists.

Hooks

The five-phase lifecycle for auth, billing, and logging.

Observability

Structured logging and OpenTelemetry tracing.

Start here

Basics

Managing Data

Features

Ecosystem

Deployment

Run it standalone

Embed it

Routes

Evaluate over HTTP

Next steps

Providers

Configuration

Hooks

Observability

​Run it standalone

​Embed it

​Routes

​Evaluate over HTTP

​Next steps

Providers

Configuration

Hooks

Observability

Run it standalone

Embed it

Routes

Evaluate over HTTP

Next steps