🔨 TagForge

Small Model Fine-tuning Toolkit

A complete toolkit for fine-tuning small language models (< 4B parameters) and deploying them anywhere - phones, edge devices, or desktop.

✨ Why TagForge?

• 🚀 2x Faster Training - Powered by Unsloth for optimized LoRA fine-tuning
• 📱 Phone Deployment - Export to ExecuTorch for iOS/Android (~50 tokens/s)
• 💻 Desktop Deployment - Export to GGUF for llama.cpp
• 📊 Auto Dataset Generation - Use LLMs to generate training data from raw text
• 📚 Built-in Data Sources - 120+ classic books from Project Gutenberg
• 🎯 Task Agnostic - Fine-tune for any task: tagging, classification, generation, etc.

📦 Supported Models

🚀 Quick Start

Installation

# Clone the repository
git clone https://github.com/yourusername/tagforge
cd tagforge

# Install dependencies
pip install -e .

# Or use the setup command
tagforge setup

Train Your First Model (5 minutes)

# Full pipeline: generate data → train → test
tagforge all --model qwen3-0.6b --num-samples 1000 --epochs 3

Custom Training

# 1. Prepare your data (JSON with text-label pairs)
tagforge prepare --input my_data.json --output data/train.json

# 2. Train
tagforge train --model qwen3-0.6b --dataset data/train.json --epochs 3

# 3. Test
tagforge test --model-path models/trained_model

# 4. Export
tagforge export gguf --model-path models/trained_model

📚 Training Data

Option 1: Use Your Own Data

Prepare a JSON file with your training examples:

[
  {
    "messages": [
      {"role": "system", "content": "You are a helpful assistant."},
      {"role": "user", "content": "What is the capital of France?"},
      {"role": "assistant", "content": "The capital of France is Paris."}
    ]
  }
]

Option 2: Generate Data with LLM

Use a larger LLM to generate training data from raw text:

# Download classic books from Project Gutenberg
tagforge download-gutenberg --num-books 50

# Generate training data using Ollama (local)
tagforge llm-generate \
  --input data/gutenberg/gutenberg_chunks.jsonl \
  --provider ollama \
  --model qwen2.5:7b

# Or use OpenAI/Anthropic
tagforge llm-generate \
  --input my_texts.txt \
  --provider openai \
  --model gpt-4o-mini

Option 3: Built-in Datasets

# Generate from built-in samples
tagforge generate --num-samples 1000

# Include public datasets (AG News, IMDB)
tagforge generate --num-samples 2000 --include-public

🛠️ Commands

🔧 Training Options

Standard LoRA Training

tagforge train \
    --model qwen3-0.6b \
    --dataset data/train.json \
    --epochs 3 \
    --lora-r 16 \
    --lora-alpha 32

Memory-Efficient Training (4-bit)

tagforge train \
    --model qwen3-0.6b \
    --load-4bit \
    --batch-size 2

Phone Deployment Training (QAT)

tagforge train \
    --model qwen3-0.6b \
    --phone-deployment \
    --epochs 3

📱 Deployment

Desktop (llama.cpp)

# Export to GGUF
tagforge export gguf --model-path models/trained_model

# With specific quantization
tagforge export gguf --model-path models/trained_model --quantization q4_k_m

# List quantization options
tagforge export gguf --list-quantizations

Phone (iOS/Android)

# Export for phone
tagforge export phone \
    --model-path models/trained_model \
    --model-family qwen3 \
    --max-context-length 1024

Performance:

📁 Project Structure

tagforge/
├── main.py                 # Main CLI
├── src/
│   ├── models/
│   │   └── registry.py     # Model registry
│   ├── data/
│   │   └── tagging_dataset_generator.py
│   └── export/
│       ├── gguf_exporter.py
│       └── phone_exporter.py
├── scripts/
│   ├── download_gutenberg.py   # Gutenberg downloader
│   ├── generate_with_llm.py    # LLM data generation
│   └── ...
└── docs/
    ├── PHONE_DEPLOYMENT.md
    ├── FINETUNING.md
    └── DEPLOYMENT.md

📋 Requirements

• Python 3.10-3.12
• CUDA-capable GPU (recommended)
• 8GB+ VRAM for training
• 16GB+ RAM

Core Dependencies

torch>=2.1.0
transformers>=4.40.0
unsloth>=2024.8
trl>=0.8.0
peft>=0.10.0

Phone Deployment (Optional)

torchao>=0.14.0
executorch
pytorch_tokenizers

🎯 Use Cases

• Text Classification - Sentiment, topic, intent classification
• Entity Extraction - Named entities, keywords, tags
• Text Generation - Summaries, completions, rewrites
• Function Calling - Tool use, API calls
• Custom Tasks - Any text-to-text task

🤝 Contributing

Contributions are welcome! Please feel free to submit a Pull Request.

📄 License

This project is licensed under the Apache 2.0 License - see the LICENSE file for details.

🙏 Acknowledgments

• Unsloth - Fast LoRA training
• ExecuTorch - Phone deployment
• llama.cpp - Desktop inference
• Project Gutenberg - Public domain books
• Google DeepMind - Gemma models
• Alibaba - Qwen models