koompi — Unified Tool for Arch Linux

One binary. AI inference, package management, system utilities, and a built-in Arch Linux assistant.

Highlights

• Real Local AI Inference — Autoregressive text generation using candle. Supports Llama, Mistral, and Phi-3 via GGUF (quantized) and SafeTensors (full-precision). OpenAI-compatible API with streaming.
• Built-in AI Assistant — Ask Arch Linux questions in natural language. Context-aware answers using your system state (disk, memory, packages, failed services).
• Package Management — Intuitive wrapper over pacman with AUR support, safety checks, orphan cleanup, and colored output.
• System Utilities — Health checks, disk/memory analysis, optimization tips.
• Flat CLI — No subcommand nesting. koompi install, koompi server, koompi ask.

Quick Start

# Build from source
cargo build --release
cp target/release/koompi /usr/local/bin/

# Initialize configuration
koompi config init

# Download a GGUF model (recommended for CPU)
koompi download TheBloke/Llama-2-7B-Chat-GGUF

# Ask the AI assistant a question
koompi ask "how to fix broken pacman keyring"

# Start the inference server
koompi server ./path/to/model.gguf --port 8000

# Install a package
koompi install firefox

# Check system health
koompi sys health

AI Inference

koompi includes a production-grade inference engine built on the candle framework:

Server

# Start with a GGUF model
koompi server ./model-q4_k_m.gguf

# Full options
koompi server ./model.gguf \
  --host 0.0.0.0 \
  --port 8000 \
  --context-length 4096 \
  --max-tokens 2048 \
  --temperature 0.7 \
  --api-key my-secret-key

OpenAI SDK Compatibility

from openai import OpenAI

client = OpenAI(
    base_url="http://localhost:8000/v1",
    api_key="my-secret-key",  # or "not-needed" if no key set
)

response = client.chat.completions.create(
    model="local",
    messages=[
        {"role": "system", "content": "You are a helpful assistant."},
        {"role": "user", "content": "Explain btrfs snapshots"},
    ],
    temperature=0.7,
    max_tokens=512,
    stream=True,
)

for chunk in response:
    if chunk.choices[0].delta.content:
        print(chunk.choices[0].delta.content, end="")

# Or use curl
curl http://localhost:8000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer my-secret-key" \
  -d '{
    "model": "local",
    "messages": [{"role": "user", "content": "Hello"}],
    "max_tokens": 256
  }'

Built-in Assistant

# Ask a question
koompi ask "how to install nvidia drivers on arch"

# Include system context (disk, memory, packages, failed services)
koompi ask "why is my system slow" --with-context

# Specify a model
koompi ask --model ./mistral.gguf "explain systemd timers"

# Interactive chat session
koompi chat

Model Management

# Download from HuggingFace
koompi download TheBloke/Mistral-7B-Instruct-v0.2-GGUF

# List downloaded models
koompi list --models

# Show AI system info (GPU, memory, SIMD, recommended models)
koompi info --ai

# Benchmark a model
koompi bench ./model.gguf --prompts 50

Package Management

# Install packages (pacman -S)
koompi install firefox vlc

# Install from AUR
koompi install google-chrome --aur

# Remove with dependencies (pacman -Rns)
koompi remove firefox --with-deps

# Full system update (pacman -Syu)
koompi update

# Update including AUR
koompi update --aur

# Search repos and AUR
koompi search firefox --all

# Clean package cache
koompi clean --aggressive

# Remove orphan packages
koompi orphan --remove

System Utilities

koompi sys info       # System information
koompi sys health     # Health check (disk, RAM, systemd, pacman DB)
koompi sys disk       # Disk usage analysis
koompi sys memory     # Memory usage analysis
koompi sys optimize   # Optimization suggestions

Configuration

koompi config init            # Create default config
koompi config set ai.port 8080
koompi config get ai.port

Config file: ~/.config/koompi/config.toml

Building

Requirements

• Rust 1.75+ (2021 edition)
• Arch Linux (primary target, works on other Linux distros)

Build

# CPU-only (default)
cargo build --release

# With CUDA GPU acceleration
cargo build --release --features cuda

# With Metal GPU acceleration (macOS)
cargo build --release --features metal

Test

cargo test  # 49 tests across 6 suites

Architecture

src/
├── main.rs                 # Entry point
├── lib.rs                  # Library root
├── cli/commands.rs         # Flat CLI definitions (clap)
├── ai/
│   ├── inference.rs        # InferenceEngine — autoregressive generation loop
│   ├── model_loader.rs     # ModelBackend — Llama/Mistral/QuantizedLlama/QuantizedPhi3
│   ├── sampler.rs          # LogitsProcessor — temperature, top-k, top-p, repeat penalty
│   ├── tokenizer.rs        # TokenizerWrapper — HuggingFace tokenizers + chat templates
│   ├── server.rs           # Axum HTTP server
│   ├── assistant.rs        # Built-in AI assistant (ask/chat)
│   ├── model.rs            # Model download, list, info, benchmark
│   ├── api/
│   │   ├── mod.rs          # OpenAI-compatible request/response types
│   │   ├── handlers.rs     # Endpoint implementations
│   │   └── routes.rs       # Router
│   └── cache/kv_cache.rs   # Standalone KV cache for custom extensions
├── pkg/                    # Package management (pacman, AUR, safety)
├── sys/                    # System utilities
├── config/                 # TOML configuration
└── utils/                  # Output formatting, progress bars

Inference Pipeline

User Prompt
    ↓
TokenizerWrapper.encode()    →  token IDs
    ↓
ModelBackend.forward()       →  logits (prefill: entire prompt)
    ↓
LogitsProcessor.sample()     →  first token
    ↓
  ┌───────────────────────────────────┐
  │  Autoregressive Decode Loop       │
  │  ModelBackend.forward(token, pos) │
  │  LogitsProcessor.sample(logits)   │
  │  Check EOS / stop sequences       │
  │  Emit token via callback          │
  └────────────┬──────────────────────┘
               ↓
TokenizerWrapper.decode()    →  generated text

License

MIT OR Apache-2.0