sageRefiner

Intelligent Context Compression Library for LLM Systems

sageRefiner provides state-of-the-art context compression algorithms to reduce token usage while maintaining semantic quality for Large Language Model applications.

Features

• 8 Compression Algorithms

  • LongRefiner: LLM-based selective compression with importance scoring
  • REFORM: Attention-based compression with KV cache optimization
  • Provence: Sentence-level pruning using DeBERTa reranker
  • LLMLingua2: Fast BERT-based token classification
  • LongLLMLingua: Question-aware perplexity-based compression
  • RECOMP-Abstractive: T5-based summarization
  • RECOMP-Extractive: BERT-based sentence selection
  • EHPC: Evaluator Heads based efficient compression

• High Compression Ratios: 2-10x compression while preserving key information

• Flexible Configuration: YAML/dict-based configuration

• Production Ready: Battle-tested in the SAGE framework

Installation

# Basic installation
pip install isage-refiner

# With benchmark support
pip install isage-refiner[benchmark]

# Development mode
git clone https://github.com/intellistream/sageRefiner.git
cd sageRefiner
pip install -e .

Quick Start

from sage_refiner import LLMLingua2Compressor

# Initialize compressor
compressor = LLMLingua2Compressor()

# Compress context
result = compressor.compress(
    context="Your long document text here...",
    question="What is the main topic?",
    target_token=500,
)

print(f"Compression rate: {result['compression_rate']:.2%}")
print(f"Compressed: {result['compressed_context']}")

Algorithms

Configuration

from sage_refiner import RefinerConfig

config = RefinerConfig(
    algorithm="llmlingua2",
    target_token=500,
    force_tokens=["important", "keyword"],
)

Examples

# Basic compression
python examples/basic_compression.py

# Compare algorithms
python examples/algorithm_comparison.py

Requirements

• Python 3.11+
• PyTorch 2.0+
• Transformers 4.43+

L3 Boundary Definition

This repository is an L3 algorithm library in the SAGE layered architecture.

• In scope
  • Context compression algorithms and their typed APIs
  • Algorithm-level orchestration, evaluation helpers, and benchmark integration
  • Model-level compute logic required by algorithm implementations
• Out of scope
  • Runtime/service wiring, networked operators, storage backends, and infrastructure adapters
  • CLI/platform orchestration responsibilities that belong to upper layers
• Forbidden imports (boundary rule)
  • Do not import from L4/L5 service/application packages in core algorithm paths
  • No backward compatibility shim/re-export/fallback layers for boundary migrations

Dependency Audit (Phase 1)

• Retained with rationale
  • torch, transformers: required by algorithm compute paths
  • numpy, pyyaml, tqdm, rich: required for algorithm runtime/config/output
• Boundary tightening completed
  • Heavy dependency usage is constrained to core algorithm/compressor modules
  • Operator fallback paths were removed in favor of fail-fast behavior

Benchmarking

The benchmarks module provides a comprehensive evaluation framework for all compression algorithms:

# Quick comparison of multiple algorithms
pip install isage-refiner[benchmark]
sage-refiner-bench compare \
    --algorithms baseline,longrefiner,reform,provence \
    --samples 100

# Detailed evaluation with budget sweep
sage-refiner-bench sweep \
    --algorithm longrefiner \
    --budgets 512,1024,2048,4096

For detailed benchmarking documentation, see benchmarks/README.md and benchmarks/STRUCTURE.md.

Citation

@software{sageRefiner2025,
  title = {sageRefiner: Context Compression for LLM},
  author = {SAGE Team},
  year = {2025},
  url = {https://github.com/intellistream/sageRefiner}
}

License

Apache License 2.0

Links

• SAGE Framework: https://github.com/intellistream/SAGE
• Issues: https://github.com/intellistream/sageRefiner/issues

Documentation & Development

• Development Guide - Setup, workflow, and guidelines
• Pre-commit Hooks - Setup and usage guide
• CI/CD Pipeline - Automated checks details

For quick setup:

bash utils/installation/quickstart.sh    # Full installation
bash utils/hooks/setup-hooks.sh          # Setup pre-commit hooks