Ollama Provider V2 for Vercel AI SDK

Use Ollama with the Vercel AI SDK, implementing the official Ollama API. This provider has minimal dependencies and is web-compatible out of the box.

Support this project — Maintained in spare time and free to use under Apache-2.0. If you value this plugin and appreciate it, a donation is appreciated and helps keep it updated with new AI SDK releases.

Why Choose Ollama Provider V2?

• ✅ Minimal Dependencies - Lean codebase with just 2 core dependencies
• ✅ Universal Compatibility - Works seamlessly in Node.js, browsers, and edge environments
• ✅ Direct Ollama Integration - Uses official Ollama API endpoints for maximum compatibility
• ✅ Advanced Features - Tool calling, streaming, thinking mode, embeddings, and completion models
• ✅ Type Safety - Full TypeScript support with comprehensive type definitions
• ✅ Zero Configuration - Works out-of-the-box with sensible defaults
• ✅ Actively Maintained - Regular updates and AI SDK v7 compatibility

Quick Start

npm install ollama-ai-provider-v2 ai

import { ollama } from 'ollama-ai-provider-v2';
import { generateText } from 'ai';

// Works everywhere - Node.js, browsers, edge environments
const { text } = await generateText({
  model: ollama('llama3.2'),
  prompt: 'Write a haiku about coding',
});

console.log(text);

Core Features

Streaming Text Generation

import { streamText } from 'ai';

const { textStream } = await streamText({
  model: ollama('llama3.2'),
  prompt: 'Tell me a story about artificial intelligence',
});

for await (const chunk of textStream) {
  process.stdout.write(chunk);
}

Tool Calling Support

AI SDK 7 defaults to a single generation step. When the model calls a tool, that step ends with empty text and populated toolCalls. Set stopWhen to allow the model to execute tools and then answer.

import { generateText, tool, stepCountIs } from 'ai';
import { z } from 'zod';

const { text, toolCalls, toolResults } = await generateText({
  model: ollama('llama3.2'),
  stopWhen: stepCountIs(5), // allow tool call + follow-up answer
  prompt: 'What is the weather like in San Francisco?',
  tools: {
    getWeather: tool({
      description: 'Get current weather for a location',
      inputSchema: z.object({
        location: z.string().describe('City name'),
        unit: z.enum(['celsius', 'fahrenheit']).optional(),
      }),
      execute: async ({ location, unit = 'celsius' }) => {
        return { temp: 18, unit, condition: 'sunny' };
      },
    }),
  },
});

Reasoning Mode (Thinking)

Unique feature for models that support chain-of-thought reasoning:

const { text } = await generateText({
  model: ollama('deepseek-r1:7b'),
  reasoning: 'high', // AI SDK 7 top-level reasoning option
  prompt: 'Solve this complex math problem step by step: 2x + 5 = 17',
});

You can also use the Ollama-specific provider option:

const { text } = await generateText({
  model: ollama('deepseek-r1:7b'),
  providerOptions: { ollama: { think: true } },
  prompt: 'Solve this complex math problem step by step: 2x + 5 = 17',
});

Advanced Ollama Options

Access Ollama's native parameters while maintaining AI SDK compatibility:

const { text } = await generateText({
  model: ollama('llama3.2'),
  providerOptions: {
    ollama: {
      options: {
        seed: 123,           // Deterministic outputs
        num_ctx: 8192,       // Context window size
        repeat_penalty: 1.1, // Control repetition
        top_k: 40,          // Advanced sampling
        min_p: 0.1,         // Minimum probability
      },
    },
  },
  prompt: 'Write a detailed analysis of Alice in Wonderland',
  temperature: 0.8, // Standard AI SDK parameters work too
});

Embeddings

import { embed } from 'ai';

// Single embedding
const { embedding } = await embed({
  model: ollama.embedding('nomic-embed-text'),
  value: 'Hello world',
});

console.log('Embedding dimensions:', embedding.length);

// Batch embeddings
const texts = ['Hello world', 'How are you?', 'AI is amazing'];
const results = await Promise.all(
  texts.map((text) =>
    embed({
      model: ollama.embedding('all-minilm'),
      value: text,
    })
  )
);

Completion Models

For use cases requiring completion-style generation:

const { text } = await generateText({
  model: ollama.completion('codellama:code'),
  prompt: 'def fibonacci(n):\n    if n <= 1:\n        return n\n    else:\n        return fibonacci(n-1) + fibonacci(n-2)\n\n# Optimize this function:\n',
});

Custom Ollama Instance

Connect to remote Ollama servers or custom configurations:

import { createOllama } from 'ollama-ai-provider-v2';

const customOllama = createOllama({
  baseURL: 'https://my-ollama-server.com/api',
  headers: {
    'Authorization': 'Bearer your-token',
  },
});

const { text } = await generateText({
  model: customOllama('llama3.2'),
  prompt: 'Hello from remote server!',
});

Supported Models

Works with any model in your Ollama installation, including:

• Chat Models: llama3.2, mistral, phi4-mini, qwen2.5, codellama, gemma3
• Vision Models: llava, llama3.2-vision, minicpm-v
• Reasoning Models: deepseek-r1:7b, deepseek-r1:1.5b, deepseek-r1:8b
• Code Models: codellama:code, codellama:python, deepseek-coder-v2
• Embedding Models: nomic-embed-text, all-minilm, mxbai-embed-large

Prerequisites

1. Ollama installed and running
2. AI SDK v7 (ai@^7)
3. Node.js 22+ (required by AI SDK 7)

# Start Ollama
ollama serve

# Pull a model
ollama pull llama3.2

Upgrading from 3.x

Version 4.0.0 adds AI SDK 7 support via the Provider V4 specification.

• Requires ai@^7 and Node.js 22+
• Drops AI SDK 5/6 support
• No changes to the public createOllama() / ollama() API surface

Contributing

Contributions are welcome! Here's how to get started:

1. Fork the repository
2. Install dependencies: pnpm install
3. Build the project: pnpm build
4. Run tests: pnpm test
5. Make your changes
6. Test locally: Copy dist/* to your project's node_modules/ollama-ai-provider-v2/dist
7. Submit a pull request

License

Apache-2.0 © nordwestt

See LICENSE.md for details.