Documentation · Package · npm

The LLM call framework. Resilience, observability, and control for every call.

One interface across OpenAI-compatible, Anthropic, Gemini, and Bedrock, with retries, circuit breaking, fallback, rate limiting, caching, and middleware built in, all running in your own process rather than a new network hop.

import Anthropic from '@anthropic-ai/sdk';
import OpenAI from 'openai';
import { fromAnthropic, fromOpenAI, VernLLM } from 'vern-llm';

const openai = fromOpenAI(new OpenAI({ apiKey: process.env.OPENAI_API_KEY }));
const anthropic = fromAnthropic(new Anthropic({ apiKey: process.env.ANTHROPIC_API_KEY }));

const llm = new VernLLM({
  client: openai,
  model: 'gpt-4o',
  fallback: { client: anthropic, model: 'claude-sonnet-5', circuitBreaker: true },
  rateLimit: { requestsPerMinute: 500, tokensPerMinute: 100_000, maxConcurrent: 20 },
  retryBudget: { windowMs: 60_000, minCalls: 20, retryRatio: 0.2 },
  maxRetries: 3,
  timeoutMs: 10_000,
  defaultMaxTokens: 1000,
  defaultReasoningEffort: 'medium',
});

const result = await llm.call({ userContent: "What's the weather in New York?" });

Works with OpenAI, Groq, Mistral, DeepSeek, Cerebras, Together AI, Fireworks AI, Ollama, Anthropic, Gemini, AWS Bedrock, or any provider reachable over HTTP via a fromFetch adapter.

Repository layout

This is a pnpm monorepo with two workspaces:

License

MIT © LakBud