A Thousand Words - Batch Captioning Tool

A powerful, customizable, and user-friendly batch captioning tool for VLM (Vision Language Models). Designed for dataset creation, this tool supports 20+ state-of-the-art models and versions, offering both a feature-rich GUI and a fully scriptable CLI commands.

Key Features

• Extensive Model Support: 20+ models including WD14, JoyTag, JoyCaption, Florence2, Qwen 2.5, Qwen 3.5, Qwen3.5-9B Gliese Abliterated, Moondream(s), Paligemma, Pixtral, smolVLM, ToriiGate).
• Batch Processing: Process entire folders and datasets in one go with a GUI or simple CLI command.
• Multi Model Batch Processing: Process the same image with several different models all at once (queued).
• Dual Interface:
  • Gradio GUI: Interactive interface for testing models, previewing results, and fine-tuning settings with immediate visual feedback.
  • CLI: Robust command-line interface for automated pipelines, scripting, and massive batch jobs.
• Highly Customizable: Extensive format options including prefixes/suffixes, token limits, sampling parameters, output formats and more.
• Customizable Input Prompts: Use prompt presets, customized prompt presets, or load input prompts from text-files or from image metadata.
• Video Captioning: Switch between Image or Video models.

---

Setup

Recommended Environment

• Python: 3.12
• CUDA: 12.8
• PyTorch: 2.8.0+cu128

Setup Instructions

1. Run the setup script:

   setup.bat
   

   This creates a virtual environment (venv), upgrades pip, and installs uv (fast package installer).

   It does not install the requirements. This need to be done manually after PyTorch and Flash Attention (optional) is installed.

   After the virtual environment creation, the setup should leave you with the virtual environment activated. It should say (venv) at the start of your console. Ensure the remaining steps is done with the virtual environment active. You can also use the venv_activate.bat script to activate the environment.

2. Install PyTorch: Visit PyTorch Get Started and select your CUDA version.

   Example for CUDA 12.8:

   pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu128
   

3. Install Flash Attention (Optional, for better performance on some models): Download a pre-built wheel compatible with your setup:

   • For Recommended Environment: For Python 3.12, Torch 2.8.0, CUDA 12.8
   • Other Versions: mjun0812's Releases
   • More Other Versions: lldacing's HuggingFace Repo

   Place the .whl file in your project folder, then install your version, for example:

   pip install flash_attn-2.8.2+cu128torch2.8-cp312-cp312-win_amd64.whl
   

4. Install Requirements:

   uv pip install -r requirements.txt
   

   Note: This requires transformers 5.x. If the install fails to resolve it from PyPI, install it manually first, then re-run:

   pip install git+https://github.com/huggingface/transformers.git
   uv pip install -r requirements.txt
   

   ⚠️ Moondream compatibility: The Moondream models (Moondream 2 and Moondream 3) require transformers <= 4.56.1 and are not compatible with transformers 5.x. On 5.x they load without error but produce garbage/empty captions. Because the newer Qwen3.5 models require transformers 5.x, the two families cannot be used from the same environment — pin transformers==4.56.1 if you need Moondream, or use a separate environment for it.

5. Launch the Application:

   gui.bat
   

   or

   py gui.py
   

6. Server Mode: To allow access from other computers on your network (and enable file zipping/downloads). This also enables the REST API on port 8000 (/api/caption, /api/models, /api/health):

   gui.bat --server
   

   or

   py gui.py --server
   

   To enable only the REST API without network exposure (localhost only):

   py gui.py --enable-api
   

---

Docker (Compose)

Prefer containers? A Dockerfile and docker-compose.yml are included. This builds a CUDA 12.8 image (Flash Attention is installed from a prebuilt wheel, no compiling) and serves the GUI + REST API together on port 8585.

Requirements

• Docker with the NVIDIA Container Toolkit (GPU passthrough).
• An NVIDIA GPU with recent drivers. On Windows, use Docker Desktop with the WSL2 backend.

Quick Start

docker compose up --build

Then open http://localhost:8585 in your browser.

Note: The build downloads PyTorch and a prebuilt Flash Attention wheel (~a few GB total), so the first build takes a while on a slow connection. There's no source compilation, and layers are cached, so rebuilds are fast.

Volumes & Models

The compose file bind-mounts these host folders into the container, so they persist and are not baked into the image:

Models download into ./models on first run and are reused afterward.

Hugging Face Token (optional)

Gated models (and higher download rate limits) need an HF token. Copy .env.example to .env next to the compose file and set it:

HF_TOKEN=hf_your_token_here

Left unset, the container runs unauthenticated.

Common Commands

docker compose up -d --build   # build and run in the background
docker compose logs -f         # follow logs
docker compose down            # stop and remove the container

---

Features Overview

Captioning

The main workspace for image and video captioning:

• Model Selection: Choose from 20+ models with good presets, information about VRAM requirements, speed, capabilities, license
• Prompt Configuration: Use preset prompt templates or create custom prompts with support for system prompts
• Custom Per-Image Prompts: Use text-files or image metadata as input prompts, or combine them with a prompt prefix/suffix for per image captioning instructions
• Generation Parameters: Fine-tune temperature, top_k, max tokens, and repetition penalty for optimal output quality
• Dataset Management: Load folders from your local drive if run locally, or drag/drop images into the dataset area
• Processing Limits: Limit the number of images to caption for quick tests or samples
• Live Preview: Interactive gallery with caption preview and manual caption editing
• Output Customization: Configure prefixes/suffixes, output formats, and overwrite behavior
• Text Post-Processing: Automatic text cleanup, newline collapsing, normalization, and loop detection removal
• Image Preprocessing: Resize images before inference with configurable max width/height
• CLI Command Generation: Generate equivalent CLI commands for easy batch processing

Multi-Model Captioning

Run multiple models on the same dataset for comparison or ensemble captioning:

• Sequential Processing: Run multiple models one after another on the same input folder
• Per-Model Configuration: Each model uses its settings from the captioning page

Tools Tab

Run various scripts and tools to manipulate and manage your files:

Augment

Augment small datasets with randomized variations:

• Crop jitter, rotation, and flip transformations
• Color adjustments (brightness, contrast, saturation, hue)
• Blur, sharpen, and noise effects
• Size constraints and forced output dimensions
• Caption file copying for augmented images

Credit: a-l-e-x-d-s-9/stable_diffusion_tools

Bucketing

Analyze and organize images by aspect ratio for training optimization:

• Automatic aspect ratio bucket detection
• Visual distribution of images across buckets
• Balance analysis for dataset quality
• Export bucket assignments

Metadata Extractor

Extract and analyze image metadata:

• Read embedded captions and prompts from image files
• Extract EXIF data and generation parameters
• Batch export metadata to text files

Resize Tool

Batch resize images with flexible options:

• Configurable maximum dimensions (width/height)
• Multiple resampling methods (Lanczos, Bilinear, etc.)
• Output directory selection with prefix/suffix naming
• Overwrite protection with optional bypass

Presets

Manage prompt templates for quick access:

• Create Presets: Save frequently used prompts as named presets
• Model Association: Link presets to specific models
• Import/Export: Share preset configurations

Settings

Configure global application defaults:

• Output Settings: Default output directory, format, overwrite behavior
• Processing Defaults: Default text cleanup options, image resizing limits
• UI Preferences: Gallery display settings (columns, rows, pagination)
• Hardware Configuration: GPU VRAM allocation, default batch sizes
• Reset to Defaults: Restore all settings to factory defaults with confirmation

Model Information

A detailed list of model properties and requirements to get an overview of what features the different models support.

Note: Minimum VRAM estimates based on quantization and optimized batch sizes. Speed measured on RTX 5090.

---

Detailed Feature Documentation

Generation Parameters

Text Processing Features

Output Options

Image Processing

• Max Width/Height: Resize images proportionally before sending to model (reduces VRAM, improves throughput)
• Visual Tokens: Control token allocation for image encoding (model-specific)

Model-Specific Features

---

Developer Guide

To add new models or features, first READ GEMINI.md. It contains strict architectural rules:

1. Config First: Defaults live in src/config/models/*.yaml. Do not hardcode defaults in Python.
2. Feature Registry: New features must optionally implement BaseFeature and be registered in src/features.
3. Wrappers: Implement BaseCaptionModel in src/wrappers. Only implement _load_model and _run_inference.

---

Example CLI Inputs

Basic Usage

Process a local folder using the standard model default settings.

python captioner.py --model smolVLM --input ./input

Input & Output Control

Specify exact paths and customize output handling.

# Absolute path input, recursive search, overwrite existing captions
python captioner.py --model wd14 --input "C:\Images\Dataset" --recursive --overwrite

# Output to specific folder, custom prefix/suffix
python captioner.py --model smolVLM2 --input ./test_images --output ./results --prefix "photo of " --suffix ", 4k quality"

Generation Parameters

Fine-tune the model creativity and length.

# Creative settings
python captioner.py --model joycaption --input ./input --temperature 0.8 --top-k 60 --max-tokens 300

# Deterministic/Focused settings
python captioner.py --model qwen3_vl --input ./input --temperature 0.1 --repetition-penalty 1.2

Model-Specific Capabilities

Leverage unique features of different architectures.

Model Versions (Size/Variant selection)

python captioner.py --model smolVLM2 --model-version 2.2B
python captioner.py --model pixtral_12b --model-version "Quantized (nf4)"

Moondream Special Modes

# Query Mode: Ask questions about the image
python captioner.py --model moondream3 --model-mode Query --task-prompt "What color is the car?"

# Detection Mode: Get bounding boxes
python captioner.py --model moondream3 --model-mode Detect --task-prompt "person"

Video Processing

# Caption videos with strict frame rate control
python captioner.py --model qwen3_vl --input ./videos --fps 4 --flash-attention

Advanced Text Processing

Clean and format the output automatically.

python captioner.py --model paligemma2 --input ./input --clean-text --collapse-newlines --strip-thinking-tags --remove-chinese

Debug & Testing

Run a quick test on limited files with console output.

python captioner.py --model smolVLM --input ./input --input-limit 4 --print-console