Cache Augmented Generation with Granite

This repository explores Cache Augmented Generation (CAG) and its usage with Granite language models, demonstrating how Granite’s extended context windows and processing power enhance efficiency by directly utilizing precomputed information.

Unlike Retrieval-Augmented Generation (RAG), which dynamically fetches relevant chunks of information in real-time followed by response generation, CAG prepares and stores all necessary information in advance.

CAG vs. RAG: Key differences

SetUp Guide

• Clone the Repository:

  git clone https://github.com/IBM/Cache-Augmented-Generation-Granite.git
  cd Cache-Augmented-Generation-Granite
  

• Install Dependencies:

  • By default, the dependencies are managed with uv Python package and project manager. Install it, if not already.

    uv sync
    

• Activate Virtual Environment:

  source .venv/bin/activate
  

  • Make sure your editor is using the correct Python virtual environment.

• Configure Environment Variables:

  cp .env.sample .env
  

  • Update .env with your Hugging Face token:

    HF_TOKEN=<your_hf_token>
    

  • You can get your Hugging Face token from Hugging Face.

• Run the Script:

  python cag_script.py
  

Reference

• Research paper: Don’t Do RAG: When Cache-Augmented Generation is All You Need for Knowledge Tasks