RAG Chat SDK ·

[!NOTE]
Recommendation from the Upstash Team

The RAG Chat SDK is designed to facilitate easier and faster prototype creation. For real projects, use Vercel AI SDK, Langchain, and LlamaIndex.

[!NOTE]
This project is a Community Project.

The project is maintained and supported by the community. Upstash may contribute but does not officially support or assume responsibility for it.

The @upstash/rag-chat package makes it easy to develop retrieval-augmented generation (RAG) chat applications with minimal setup and configuration.

Features:

• Next.js compatibility with streaming support
• Ingest entire websites, PDFs, and more out of the box
• Built-in vector store for your knowledge base
• (Optional) built-in Redis compatibility for fast chat history management
• (Optional) built-in rate limiting
• (Optional) disableRag option to use it as LLM + chat history
• (Optional) Analytics via Helicone, Langsmith, and Cloudflare AI Gateway

Getting Started

Installation

Install the package using your preferred package manager:

pnpm add @upstash/rag-chat

bun add @upstash/rag-chat

npm i @upstash/rag-chat

Quick Start

1. Set up your environment variables:

UPSTASH_VECTOR_REST_URL="XXXXX"
UPSTASH_VECTOR_REST_TOKEN="XXXXX"


# if you use OpenAI compatible models
OPENAI_API_KEY="XXXXX"

# or if you use Upstash hosted models
QSTASH_TOKEN="XXXXX"

# Optional: For Redis-based chat history (default is in-memory)
UPSTASH_REDIS_REST_URL="XXXXX"
UPSTASH_REDIS_REST_TOKEN="XXXXX"

2. Initialize and use RAGChat:

import { RAGChat } from "@upstash/rag-chat";

const ragChat = new RAGChat();

const response = await ragChat.chat("Tell me about machine learning");
console.log(response);

Basic Usage

import { RAGChat, openai } from "@upstash/rag-chat";

export const ragChat = new RAGChat({
  model: openai("gpt-4-turbo"),
});

await ragChat.context.add({
  type: "text",
  data: "The speed of light is approximately 299,792,458 meters per second.",
});

await ragChat.context.add({
  type: "pdf",
  fileSource: "./data/physics_basics.pdf",
});
const response = await ragChat.chat("What is the speed of light?");

console.log(response.output);

Docs

General

• Getting Started
• How-To
• API
• Config
• Debug
• Features

Integrations

• Anthropic
• Custom
• GROQ
• Helicone
• LangSmith
• Mistral AI
• Next.js
• Ollama
• Open Router
• OpenAI
• Overview
• Together AI
• Unstructured
• Vercel AI

Quickstarts

• Cloudflare Workers
• Hono
• Next.js
• Next.js Server Actions
• Node.js
• Nuxt
• Overview
• SvelteKit