ai-gateway-provider

AI Gateway Provider for Vercel AI SDK

This library provides an AI Gateway Provider for the Vercel AI SDK, enabling you to seamlessly integrate multiple AI models from different providers behind a unified interface. It leverages Cloudflare's AI Gateway to manage and optimize your AI model usage.

Features

• Runtime Agnostic: Works in all JavaScript runtimes supported by the Vercel AI SDK including Node.js, Edge Runtime, and more.
• Automatic Provider Fallback: ✨ Define an array of models and the provider will automatically fallback to the next available provider if one fails, ensuring high availability and resilience for your AI applications.

Installation

npm install ai-gateway-provider

Usage

Basic Example with API Key

import { createAiGateway } from "ai-gateway-provider";
import { createOpenAI } from "ai-gateway-provider/providers/openai";
import { generateText } from "ai";

const aigateway = createAiGateway({
	accountId: "{CLOUDFLARE_ACCOUNT_ID}",
	gateway: "{GATEWAY_NAME}",
	apiKey: "{CF_AIG_TOKEN}", // If your AI Gateway has authentication enabled
});

const openai = createOpenAI({ apiKey: "{OPENAI_API_KEY}" });

const { text } = await generateText({
	model: aigateway(openai.chat("gpt-5.1")),
	prompt: "Write a vegetarian lasagna recipe for 4 people.",
});

Basic Examples with BYOK / Unified Billing

import { createAiGateway } from "ai-gateway-provider";
import { createOpenAI } from "ai-gateway-provider/providers/openai";
import { generateText } from "ai";

const aigateway = createAiGateway({
	accountId: "{CLOUDFLARE_ACCOUNT_ID}",
	gateway: "{GATEWAY_NAME}",
	apiKey: "{CF_AIG_TOKEN}",
});

const openai = createOpenAI();

const { text } = await generateText({
	model: aigateway(openai.chat("gpt-5.1")),
	prompt: "Write a vegetarian lasagna recipe for 4 people.",
});

Unified API / Dynamic Routes

import { createAiGateway } from "ai-gateway-provider";
import { unified, createUnified } from "ai-gateway-provider/providers/unified";
import { generateText } from "ai";

const aigateway = createAiGateway({
	accountId: "{{CLOUDFLARE_ACCOUNT_ID}}",
	gateway: "{{GATEWAY_NAME}}",
	apiKey: "{{CF_AIG_TOKEN}}",
});

const { text } = await generateText({
	model: aigateway(unified("dynamic/customer-support")),
	prompt: "Write a vegetarian lasagna recipe for 4 people.",
});

Automatic Fallback Example

// Define multiple provider options with fallback priority
const model = aigateway([
	anthropic("claude-3-5-haiku-20241022"), // Primary choice
	openai.chat("gpt-4o-mini"), // First fallback
	mistral("mistral-large-latest"), // Second fallback
]);

// The system will automatically try the next model if previous ones fail
const { text } = await generateText({
	model,
	prompt: "Suggest three names for my tech startup.",
});

Cloudflare AI Binding Example

Binding Benefits:

• Faster Requests: Saves milliseconds by avoiding open internet routing.
• Enhanced Security: Uses a special pre-authenticated pipeline.
• No Cloudflare API Token Required: Authentication is handled by the binding.

const aigateway = createAiGateway({
	binding: env.AI.gateway("my-gateway"),
	options: {
		// Optional per-request override
		skipCache: true,
	},
});
const openai = createOpenAI({ apiKey: "openai api key" });
const anthropic = createAnthropic({ apiKey: "anthropic api key" });

const model = aigateway([
	anthropic("claude-3-5-haiku-20241022"), // Primary choice
	openai.chat("gpt-4o-mini"), // Fallback if first fails
]);

const { text } = await generateText({
	model: model,
	prompt: "Write a vegetarian lasagna recipe for 4 people.",
});

Request-Level Options

You can now customize AI Gateway settings for each request:

const aigateway = createAiGateway({
	// ... other configs

	options: {
		// all fields are optional!
		cacheKey: "my-custom-cache-key",
		cacheTtl: 3600, // Cache for 1 hour
		skipCache: false,
		metadata: {
			userId: "user123",
			requestType: "recipe",
		},
		retries: {
			maxAttempts: 3,
			retryDelayMs: 1000,
			backoff: "exponential",
		},
	},
});

Configuration

createAiGateway(options: AiGatewaySettings)

API Key Authentication

• accountId: Your Cloudflare account ID
• gateway: The name of your AI Gateway
• apiKey (Optional): Your Cloudflare API key

Cloudflare AI Binding

• binding: Cloudflare AI Gateway binding
• options (Optional): Request-level AI Gateway settings

Request Options

• cacheKey: Custom cache key for the request
• cacheTtl: Cache time-to-live in seconds
• skipCache: Bypass caching for the request
• metadata: Custom metadata for the request
• collectLog: Enable/disable log collection
• eventId: Custom event identifier
• requestTimeoutMs: Request timeout in milliseconds
• retries: Retry configuration
  • maxAttempts: Number of retry attempts (1-5)
  • retryDelayMs: Delay between retries
  • backoff: Retry backoff strategy ('constant', 'linear', 'exponential')

Supported Providers

• OpenAI
• Anthropic
• DeepSeek
• Google AI Studio
• Grok
• Mistral
• Perplexity AI
• Replicate
• Groq

Supported Methods

Currently, the following methods are supported:

• Non-streaming text generation: Using generateText() from the Vercel AI SDK
• Chat completions: Using generateText() with message-based prompts

More can be added, please open an issue in the GitHub repository!

Error Handling

The library throws the following custom errors:

• AiGatewayUnauthorizedError: Your AI Gateway has authentication enabled, but a valid API key was not provided.
• AiGatewayDoesNotExist: Specified AI Gateway does not exist

License

This project is licensed under the MIT License - see the LICENSE file for details.

Relevant Links

• Vercel AI SDK Documentation
• Cloudflare AI Gateway Documentation
• GitHub Repository