Transfer a local ollama model to a remote ollama server
C#
15
54 commits
updated Oct 3, 2026
README
About
osync is a powerful command-line tool for managing Ollama models across local and remote servers.
Key Features
- 🖥️ Interactive TUI - Full-screen terminal interface with keyboard shortcuts for all operations
- 🚀 Fast Model Transfer - Copy models between local and remote Ollama servers with high-speed uploads
- 🔄 Remote-to-Remote Copy - Transfer models directly between remote servers with memory-buffered streaming
- 📋 Smart Model Management - List, copy, rename, and delete models with pattern matching support
- 🔄 Incremental Uploads - Skips already transferred layers, saving bandwidth and time
- 📊 Progress Tracking - Real-time progress bars with transfer speed indicators
- 🏷️ Auto-tagging - Automatically applies
:latesttag when not specified - 🔍 Advanced Listing - Sort models by name, size, or modification time with visual indicators
- 🌐 Multi-registry Support - Works with registry.ollama.ai, hf.co, and custom registries
- 💾 Offline Deployment - Perfect for air-gapped servers and isolated networks
- 🎯 Wildcard Patterns - Use
*wildcards for batch operations - ⚡ Bandwidth Control - Throttle upload speeds and configure memory buffer size
- 🎨 Theme Support - 34 true-color themes (Dracula, Nord, Tokyo Night, Catppuccin, Gruvbox, Rose Pine, Kanagawa, Solarized, light themes, ...) for
manageand the colored command output, adapted to 256 and 16-color terminals - 💬 Interactive Chat - Chat with models directly from the CLI
- 🧠 Memory Management - Load/unload models from VRAM with process status monitoring
- 📊 Quantization Comparison - Compare quality and performance across model quantizations with detailed scoring
- 🔬 Context Benchmark - Test model context tracking with dynamic story-based benchmarks, tools, custom test suites
- 📈 Real-Time Monitor - Live dashboard with GPU/CPU metrics, VRAM usage graphs, model status, and Ollama process tracking
- 🔀 Ollama and xOllama - Works with Ollama and the xOllama fork (see Local server)
Built With
[C# .NET 10]
Getting Started
Prerequisites
[Windows/Linux/MacOS]
[Arm64/x64/Mac]
Installation
[Download latest binary release]
[Build from sources]
Clone the repo
dotnet build(any OS, .NET 10 SDK) or Visual Studio 2022;dotnet publish osync/osync.csproj -c Release -r <win-x64|linux-x64|osx-arm64|osx-x64>for a single-file executable. See docs/DEVELOPMENT.md for tests, CI and releases.
Local server (Ollama and xOllama)
Commands without -d work on the local server, found in this order:
XOLLAMA_HOST(xOllama), thenOLLAMA_HOST(Ollama); bind addresses such as0.0.0.0are mapped tolocalhost. With"ignoreEnvironment": true(osync setup server env ignore, or the questionosync setup serverasks when one of them is set) the configured server comes first- The server in the preferences file (
osync setup server) - Otherwise the first server answering on
localhost:11434(Ollama default) orlocalhost:22434(xOllama default)
osync install configures it: when exactly one server answers on this machine it is used without questions; when none or both answer, it asks for the server type (1 Ollama, 2 xOllama, 3 both side by side), host and ports. With both, one is the default local server and each gets an alias (ollama, xollama), so the other one is always at hand: osync ls -d xollama, osync cp model xollama/. Change it any time with osync setup server.
osync detects whether a server is Ollama or xOllama (osync ps shows it) and, for local operations, runs the matching CLI: xollama when the local server is xOllama (or only xollama is installed), else ollama. Set OSYNC_OLLAMA_CLI to force a specific CLI. The models directory is taken from XOLLAMA_MODELS, then OLLAMA_MODELS, then the platform default.
Remote servers given without a port use :11434, or :22434 when the host only answers there (xOllama).
Preferences file
osync keeps user preferences in settings.json in the per-OS configuration folder (osync -v --verbose shows the path):
| OS | Location |
|---|---|
| Windows | %APPDATA%\osync\settings.json |
| macOS | ~/Library/Application Support/osync/settings.json |
| Linux | $XDG_CONFIG_HOME/osync/settings.json (default ~/.config/osync/settings.json) |
{
"server": { "flavor": "ollama", "host": "localhost", "both": true },
"aliases": {
"ollama": "http://localhost:11434",
"xollama": "http://localhost:22434",
"gpu": "http://192.168.1.10:11434"
},
"colorMode": "auto",
"manage": { "theme": "Dracula", "sort": "size-", "servers": [ "gpu" ] },
"shell": { "theme": "Tokyo Night" }
}
server- the local server:flavor(auto,ollama,xollama),hostandport(default: the flavor's port);bothwhen Ollama and xOllama run side by side (flavoris then the default one).XOLLAMA_HOST/OLLAMA_HOSToverride it, unlessignoreEnvironmentistrue.aliases- server aliases.colorMode-auto(detect),truecolor,256,16ornone;OSYNC_COLOR_MODEandNO_COLORoverride it.manage.theme,manage.sort- theme (also chosen with Ctrl+T inmanage) and initial sort order (name+,name-,size-,size+,created-,created+) ofosync manage.manage.servers- aliasesmanageswitches to with Ctrl+Left / Ctrl+Right, after the local server (with Ollama and xOllama side by side the other one is always included).shell.theme- colors of the command output (any theme name, orplainfor no colors).OSYNC_CONFIG_DIRmoves the settings folder.
Everything can be changed with osync setup instead of editing the file.
Server aliases
An alias is a short name for a server (osync setup alias add gpu 192.168.1.10), usable wherever osync expects a server:
osync ls -d gpu # list the models of the server
osync cp qwen3:8b gpu/ # upload
osync cp gpu/qwen3:8b qwen3-gpu # download
osync manage gpu
An alias takes precedence over a model namespace with the same name (gpu/model). Names start with a letter and use letters, digits, - and _.
Colors
osync detects the terminal's color depth from COLORTERM (truecolor/24bit), TERM (*-256color, *-direct), TERM_PROGRAM and Windows Terminal, and uses true color, 256 or 16 colors accordingly (osync -v --verbose shows what was detected and why). SSH does not forward COLORTERM by default, so terminals that support true color are seen as 256-color over SSH/tmux: set "colorMode": "truecolor" in the settings file, or OSYNC_COLOR_MODE=truecolor, or forward COLORTERM (SendEnv COLORTERM / AcceptEnv COLORTERM). NO_COLOR disables colors.
Command output is colored with the shell theme (osync setup shell theme NAME; osync setup shell themes shows every theme with a preview): the same 34 themes as manage, with light themes for light terminals (the default follows COLORFGBG when the terminal sets it), or plain. Output that goes to a pipe or a file is never colored.
osync manage draws with 24-bit colors; at 256 colors every theme color is snapped to the xterm-256 palette (tmux and most 256-color terminals then show exactly that color), at 16 colors the themes switch to the 16 standard colors with contrast checks. macOS Terminal.app gets 16 colors unless colorMode says truecolor (it misreads 24-bit colors on older macOS). If manage does not start or draw correctly on an old Windows console, OSYNC_TUI_DRIVER=windows selects the Windows console driver (ansi and dotnet are the others).
Usage
DeepWiki
An AI generated DeepWiki is available here:
https://deepwiki.com/mann1x/osync
Quick Start
# Interactive TUI for model management (recommended)
osync manage
# Copy local model to remote server
osync cp llama3 http://192.168.100.100:11434
# Copy to remote server (short forms - port 11434 and http:// are defaults)
osync cp llama3 192.168.100.100 # IP address detected as server
osync cp llama3 myserver:11434 # hostname with port
osync cp llama3 myserver/ # trailing slash indicates server
# List all local models
osync ls
# Chat with a model
osync run llama3
# Interactive REPL mode with tab completion
osync
Commands
Copy (cp)
Copy models locally, to remote servers, or between remote servers.
# Local copy (create backup)
osync cp llama3 my-backup-llama3
osync cp llama3:70b llama3:backup-v1
# Local to remote (upload to server) - multiple ways to specify server
osync cp llama3 http://192.168.0.100:11434 # Full URL
osync cp llama3 192.168.0.100 # IP address (auto: http:// + :11434)
osync cp llama3 192.168.0.100:11434 # IP with port (auto: http://)
osync cp llama3 myserver:11434 # Hostname with port
osync cp llama3 myserver/ # Trailing slash = server
# Copy HuggingFace model to remote (uses source model name)
osync cp hf.co/unsloth/gemma-3-1b-it-GGUF:Q4_K_M 192.168.0.100
# Remote to remote (transfer between servers)
osync cp http://192.168.0.100:11434/qwen2:7b http://192.168.0.200:11434/qwen2:latest
osync cp http://server1:11434/llama3 http://server2:11434/llama3-copy
# Remote to local (download from a server to the local server)
osync cp http://192.168.0.100:11434/my-finetune my-finetune
# With custom memory buffer size (default: 512MB)
osync cp http://server1:11434/llama3 http://server2:11434/llama3 -BufferSize 256MB
osync cp http://server1:11434/qwen2 http://server2:11434/qwen2 -BufferSize 1GB
# With bandwidth throttling
osync cp llama3 http://192.168.0.100:11434 -bt 50MB
Features:
- Automatic
:latesttag when not specified - Prevents overwriting existing models
- Skips already uploaded layers
- Real-time progress with transfer speed
- Memory-buffered streaming for remote-to-remote transfers
- Simultaneous download and upload with backpressure control
- Smart server detection: IP addresses,
hostname:port, orhostname/auto-detected as remote servers - Uses source model name when destination server has no model specified
Remote-to-remote and remote-to-local copies (push relay):
Ollama has no API to download a model, but a server can push a model to a registry. For copies out of a server, osync runs a temporary registry endpoint (the relay) on the machine where osync runs:
- the source server pushes the model to the relay under a temporary name;
- the relay streams every blob straight into the destination server (no disk use, memory bounded by
-BufferSize,-btthrottling applies); blobs the destination already has are skipped; - the destination installs the model's manifest from the relay, so the copy is complete: weights, projector, template, parameters, system prompt, license, messages;
- the temporary names are deleted on both servers.
This copies any model the source has, including models you created or imported, works between Ollama and xOllama in both directions, and needs no internet access.
Requirements and options:
- The source server must be able to connect to the machine running osync (and the destination should too, to install the manifest; otherwise osync recreates the model from its files). The relay listens on a free port chosen by the OS (above 1024), or on
OSYNC_RELAY_PORT; open your firewall accordingly. A Windows destination cannot store a model named after the relay'shost:port, so the model is recreated there from its manifest (byte for byte); a Windows source needs the relay on port 80, which osync then uses for that copy if it is free. OSYNC_RELAY_HOST- address the servers should use to reach osync (default: the local address that routes to the source server), e.g. behind NATOSYNC_RELAY_PORT- fixed relay port (e.g. one opened in the firewall)OSYNC_RELAY_INSTALL=create- skip installing the manifest from the relay and recreate the model with/api/create(for destinations that cannot connect to osync)- If the source server cannot reach the relay, osync falls back to downloading the blobs from registry.ollama.ai, which only works for models pulled from the Ollama registry.
List (ls)
List models with filtering and sorting options.
# List all local models
osync ls
# List models matching pattern
osync ls "llama*"
osync ls "*:7b"
osync ls "mannix/*"
# List remote models
osync ls http://192.168.0.100:11434 # Full URL
osync ls 192.168.0.100 # IP address (auto: http:// + :11434)
osync ls myserver/ # Hostname with trailing slash
osync ls "qwen*" -d 192.168.0.100 # Filter with pattern on remote
# Sort by size (descending)
osync ls --size
# Sort by size (ascending)
osync ls --sizeasc
# Sort by modified time (newest first)
osync ls --time
# Sort by modified time (oldest first)
osync ls --timeasc
Output:
NAME ID SIZE MODIFIED
llama3:latest 365c0bd3c000 5 GB 2 months ago
qwen2:7b 648f809ced2b 4 GB 1 years ago
mistral:latest 2ae6f6dd7a3d 4 GB 1 years ago
Rename (rename, mv, ren)
Rename models safely by copying and deleting the original.
# Rename with implicit :latest tag
osync rename llama3 my-llama3
osync mv llama3 my-llama3
# Rename with explicit tags
osync ren llama3:7b my-custom-llama:v1
# Create versioned backup
osync mv qwen2 qwen2:backup-20241218
Features:
- Three-step process: copy → verify → delete
- Checks if destination already exists
- Only deletes original after successful verification
Remove (rm, delete, del)
Delete models with pattern matching.
# Delete specific model
osync rm tinyllama
osync rm llama3:7b
# Delete with pattern
osync rm "test-*"
osync rm "*:backup"
# Delete from remote server
osync rm "old-model*" http://192.168.0.100:11434
Features:
- Automatic
:latesttag fallback - Wildcard pattern support
- Confirmation before deletion
- Works on local and remote servers
Update (update)
Update models to their latest versions locally or on remote servers.
# Update all local models
osync update
osync update "*"
# Update specific model
osync update llama3
osync update llama3:latest
# Update models matching pattern
osync update "llama*"
osync update "*:7b"
osync update "hf.co/unsloth/*"
# Update all models on remote server
osync update http://192.168.0.100:11434
osync update "*" http://192.168.0.100:11434
# Update specific models on remote server
osync update "llama*" http://192.168.0.100:11434
Features:
- Updates models to latest available versions
- Shows real-time progress during updates
- Indicates whether each model was updated or already up to date
- Supports wildcard pattern matching
- Works on both local and remote servers
- Default pattern is
*(all models) when not specified
Output:
Updating 2 model(s)...
Updating 'llama3:latest'...
pulling manifest
pulling 6a0746a1ec1a... 100% ▕████████████████▏ 4.7 GB
✓ 'llama3:latest' updated successfully
Updating 'qwen2:7b'...
✓ 'qwen2:7b' is already up to date
Show (show)
Display detailed information about a model.
# Show information about a local model
osync show llama3
osync show qwen2:7b
# Show information about a remote model
osync show llama3 -d http://192.168.0.100:11434
# Show specific information
osync show llama3 --license # Show license only
osync show llama3 --modelfile # Show modelfile only
osync show llama3 --parameters # Show parameters only
osync show llama3 --system # Show system prompt only
osync show llama3 --template # Show template only
osync show llama3 -v # Show all information
Options:
<model>- Model name to show information (required)-d <url>- Remote server URL (default: local)--license- Show license information--modelfile- Show modelfile--parameters- Show parameters--system- Show system prompt--template- Show template-v, --verbose- Show all information
Features:
- Displays model metadata and configuration
- Shows model file details, parameters, and system information
- Works on both local and remote servers
Pull (pull)
Pull a model from the Ollama registry.
# Pull a model from the registry
osync pull llama3
osync pull qwen2:7b
osync pull hf.co/unsloth/llama3
# Pull to a remote server
osync pull llama3 http://192.168.0.100:11434
Features:
- Downloads models from Ollama registry
- Shows real-time progress during download
- Supports library models and user models
- Works on both local and remote servers
Run (run, chat)
Interactive chat with a model.
# Chat with a local model
osync run llama3
osync chat qwen2:7b
# Chat with a model on remote server
osync run llama3 -d http://192.168.0.100:11434
# With extended thinking mode (for reasoning models)
osync run qwen3 --think medium
# With verbose output (shows timing stats)
osync run llama3 --verbose
# Vision models with custom image dimensions
osync run llava --dimensions 512
Options:
<model>- Model name to run/chat with (required)-d <url>- Remote server URL (default: local)--format <value>- Response format (e.g., json)--keepalive <duration>- Keep alive duration (e.g., 5m, 1h, default: server default)--nowordwrap- Disable word wrap--verbose- Show verbose output (timing stats)--dimensions <value>- Image dimensions for vision models (e.g., 512)--hidethinking- Hide thinking process output--insecure- Allow insecure connections--think <level>- Enable extended thinking (reasoning) mode with level (low, medium, high)--truncate- Truncate long context (default: server setting)
Features:
- Interactive conversation mode
- Preloads model into memory before chat
- Shows model loading status
- Extended thinking support for reasoning models (qwen3, deepseek-r1, etc.)
- Type
/byeor press Ctrl+D to exit - Works on both local and remote servers
Process Status (ps)
Show models currently loaded in memory, plus system monitoring when running locally.
# Show loaded models on local server (includes GPU and process stats)
osync ps
# Show loaded models on remote server (no GPU/process stats)
osync ps http://192.168.0.100:11434 # Full URL
osync ps 192.168.0.100 # IP address (auto: http:// + :11434)
osync ps myserver/ # Hostname with trailing slash
osync ps -d myserver:11434 # Using -d flag with port
Features:
- Displays models loaded in VRAM
- Shows VRAM usage with percentage when partially loaded
- Displays model size, context length, and expiration time
- Formatted table output
- Local only: Ollama process CPU and memory usage
- Local only: GPU monitoring (NVIDIA via nvidia-smi, AMD via rocm-smi)
Output (local server):
Loaded Models:
---------------------------------------------------------------------------------------------------------------------------------------
NAME ID SIZE VRAM USAGE CONTEXT UNTIL
---------------------------------------------------------------------------------------------------------------------------------------
tinyllama:1.1b-chat-v1-fp16 71c2f9b69b52 2.11 GB (1B) 1.33 GB (63%) 4096 4 minutes from now
---------------------------------------------------------------------------------------------------------------------------------------
Ollama Process:
PID: 12345 CPU: 2.3% Memory: 1.2 GB (Working Set)
GPU Status (NVIDIA):
GPU 0: NVIDIA GeForce RTX 4090
Utilization: 45% Memory: 8192 MB / 24576 MB (33%) Temp: 65°C Power: 250 W / 450 W
GPU Monitoring:
- Automatically detects nvidia-smi (NVIDIA) or rocm-smi (AMD)
- Shows GPU utilization, memory usage, temperature, and power draw
- Supports multiple GPUs
Real-Time Monitor (psmonitor, monitor, psm)
Interactive real-time monitoring dashboard inspired by nvitop, with braille-dot graphs and color-coded metrics.
# Start monitor with default settings (5s refresh, 5m history)
osync psmonitor
osync monitor
osync psm
# Custom refresh interval (2 seconds)
osync psmonitor 2s
osync psmonitor -L 2
# Custom history duration (10 minutes) - multiple formats supported
osync psmonitor -Hi 10 # Plain integer = minutes (10 minutes)
osync psmonitor -Hi 10m # Explicit minutes
osync psmonitor --history 30m # 30 minutes
osync psmonitor -Hi 1h # 1 hour
osync psmonitor -Hi 1h30m # 1 hour 30 minutes
# Combined options
osync psmonitor -L 2 -Hi 15 # 2s refresh, 15 minutes history
# Monitor remote server (shows loaded models only, no GPU/system stats)
osync psmonitor -d http://192.168.0.100:11434
Options:
-d <url>- Remote server URL (default: local). Remote mode shows loaded models only, no GPU/system stats-L <interval>- Refresh interval (default: 5s). Supports: 5, 5s, 30s, 1m, 5m, 1h30m-Hi, --history <duration>- Initial graph history duration (default: 5m). Plain integers are treated as minutes (e.g., -Hi 10 = 10 minutes). Supports: 1m, 30m, 1h, 1h30m
Features:
- Real-time GPU utilization graphs with braille characters
- CPU usage history graph
- Color-coded metrics (green < 50%, yellow 50-80%, red > 80%)
- GPU metrics: utilization, VRAM, temperature, power, clocks
- System metrics: CPU, RAM, Swap with progress bars
- Loaded Ollama models with expiration times
- NVIDIA driver and CUDA version display
Interactive Controls:
- Q / Esc - Quit monitor
- P - Pause/Resume updates
- H - Show help
- ←/→ - Adjust graph history (+/- 1 minute, range: 1-60m)
- ↑/↓ - Adjust refresh interval (+/- 1 second)
- Ctrl+C - Exit
Windows Terminal Font Requirements:
The monitor uses Unicode braille characters (U+2800-U+28FF) for graphs. On Windows, you need a font that supports these characters:
-
Recommended fonts:
- Cascadia Code / Cascadia Mono (included with Windows Terminal)
- DejaVu Sans Mono
- Consolas (partial support)
- NSimSun
-
To change font in Windows Terminal:
- Open Windows Terminal settings (Ctrl+,)
- Select your profile (e.g., "Command Prompt" or "PowerShell")
- Go to "Appearance" → "Font face"
- Select "Cascadia Code" or "Cascadia Mono"
-
Note: CMD with default raster fonts will show "?" for braille characters. Use Windows Terminal for best results.
Load (load)
Preload a model into memory.
# Load a model on local server
osync load llama3
osync load qwen2:7b
# Load a model on remote server
osync load llama3 -d http://192.168.0.100:11434
# URL format with embedded model name
osync load http://192.168.0.100:11434/llama3
Options:
<model>- Model name to load into memory (required)-d <url>- Remote server URL (default: local)
Features:
- Preloads model into VRAM for faster inference
- Shows elapsed time and API-reported load duration
- Works on both local and remote servers
Unload (unload)
Unload a model from memory.
# Unload a specific model on local server
osync unload llama3
osync unload qwen2:7b
# Unload all models (no model name specified)
osync unload
# Unload a model on remote server
osync unload llama3 -d http://192.168.0.100:11434
# URL format with embedded model name
osync unload http://192.168.0.100:11434/llama3
Options:
<model>- Model name to unload from memory (optional - if not specified, unloads all models)-d <url>- Remote server URL (default: local)
Features:
- Frees VRAM by unloading model
- Immediate unloading (keep-alive set to 0)
- Works on both local and remote servers
Quantization Comparison (qc)
Run comprehensive tests comparing quantization quality and performance across model variants.
# Compare quantizations of a model (f16 as base)
osync qc -M llama3.2 -Q q4_k_m,q5_k_m,q8_0
# Specify custom base quantization
osync qc -M llama3.2 -Q q4_k_m,q5_k_m -B fp16
# Test on remote server (multiple ways to specify)
osync qc -M llama3.2 -Q q4_k_m,q5_k_m -D http://192.168.1.100:11434
osync qc -M llama3.2 -Q q4_k_m,q5_k_m -D 192.168.1.100 # IP (auto: http:// + :11434)
osync qc -M llama3.2 -Q q4_k_m,q5_k_m -D myserver/ # trailing slash
# Custom output file
osync qc -M llama3.2 -Q q4_k_m,q5_k_m -O my-results.json
# Adjust test parameters
osync qc -M llama3.2 -Q q4_k_m,q5_k_m -Te 0.1 -S 42 -To 0.9
How It Works:
The qc command runs a comprehensive test suite (50 questions across 5 categories: Reasoning, Math, Finance, Technology, Science) on each quantization variant and captures detailed metrics using Ollama's logprobs API.
Scoring Algorithm:
Each quantization is compared against the base model using a 4-component weighted scoring system:
-
Token Sequence Similarity (5% weight)
- Uses Longest Common Subsequence (LCS) to measure token sequence matching
- More forgiving of small variations while detecting major differences
- Higher score = quantization produces similar token sequences as base
-
Logprobs Divergence (70% weight)
- Compares sequence-level confidence between base and quantization
- Calculates average confidence (mean logprob) for each sequence
- Formula:
100 × exp(-confidence_difference × 2) - Higher score = quantization has similar confidence in its predictions
-
Answer Length Consistency (5% weight)
- Compares total token count between base and quantization answers
- Formula:
100 × exp(-2 × |1 - length_ratio|) - Higher score = quantization produces similar-length answers
-
Perplexity Score (20% weight)
- Compares model confidence via perplexity (lower = more confident)
- Perplexity:
exp(-average_logprob) - Formula:
100 × exp(-0.5 × |1 - perplexity_ratio|) - Higher score = quantization maintains similar confidence levels
Overall Confidence Score: Weighted sum of all four components (0-100%)
Color Coding:
- Green (90-100%): Excellent quality preservation
- Lime (80-90%): Very good quality
- Yellow (70-80%): Good quality
- Orange (50-70%): Moderate quality loss
- Red (below 50%): Significant quality degradation
Performance Metrics:
- Evaluation tokens per second (generation speed)
- Prompt tokens per second (encoding speed)
- Performance percentages vs base model
Results File:
Results are saved as JSON (modelname.qc.json by default) with:
- Test suite name and version
- Test options (temperature, seed, top_p, top_k, etc.)
- Per-quantization results with all question answers and tokens
- Model metadata (family, parameter size, quantization type, enhanced quantization with tensor analysis, disk size)
- Model digest (SHA256 and short digest for model identification/verification)
Incremental Testing:
You can add new quantizations to existing results without re-testing:
# Initial test
osync qc -M llama3.2 -Q q4_k_m,q5_k_m
# Later add more quantizations (f16 and q8_0 will be skipped if already tested)
osync qc -M llama3.2 -Q q8_0,q6_k
# Force re-run testing for quantizations already in results file
osync qc -M llama3.2 -Q q4_k_m,q5_k_m --force
Resume Support:
Testing can be interrupted with Ctrl+C and resumed later:
# Start testing (press Ctrl+C, then 'y' to confirm save and exit)
osync qc -M llama3.2 -Q q4_k_m,q5_k_m,q8_0
# Resume from where you left off
osync qc -M llama3.2 -Q q4_k_m,q5_k_m,q8_0
When resuming:
- Already-completed quantizations are skipped
- Partially-tested quantizations continue from the last saved question
- Missing judgments are automatically detected and run
- Missing model digests are automatically retrieved and backfilled
- Progress bars show resumed position
Cancellation & Timeout Handling:
- First Ctrl+C prompts for confirmation (y=cancel and save, n=continue)
- Second Ctrl+C force exits immediately
- Timeouts are retried automatically with exponential backoff
- After retry exhaustion, prompts: y=cancel, n=double timeout and retry
- On-demand models with incomplete results are preserved for resume
Wildcard Tag Selection:
Use wildcards (*) to select multiple quantizations from HuggingFace or Ollama registries:
# Test all quantizations from a HuggingFace repository
osync qc -M LFM2.5-1.2B-Instruct -b F16 -Q hf.co/LiquidAI/LFM2.5-1.2B-Instruct-GGUF:*
# Test only IQ quantizations (case-insensitive)
osync qc -M hf.co/LiquidAI/LFM2.5-1.2B-Instruct-GGUF -b F16 -Q IQ*
# Test Q4 and Q5 quantizations
osync qc -M llama3.2 -b fp16 -Q Q4*,Q5*
# Mix HuggingFace base with Ollama quants
osync qc -M LFM2.5-1.2B-Instruct -b hf.co/LiquidAI/LFM2.5-1.2B-Instruct-GGUF:F16 -Q IQ*,Q4*,Q5*
# Multiple HuggingFace patterns
osync qc -M LFM2.5-1.2B-Instruct -b hf.co/LiquidAI/LFM2.5-1.2B-Instruct-GGUF:F16 -Q hf.co/LiquidAI/LFM2.5-1.2B-Instruct-GGUF:IQ*,hf.co/LiquidAI/LFM2.5-1.2B-Instruct-GGUF:Q5*
Wildcard behavior:
*matches any characters (case-insensitive)- Tags are fetched from HuggingFace API for
hf.co/...models - Tags are scraped from Ollama website for Ollama registry models
- Patterns without model prefix use the
-Mmodel as source
Options:
-M <name>- Model name without tag (required)-Q <tags>- Comma-separated quantization tags to compare, supports wildcards (e.g.,Q4*,IQ*) (required)-B <tag>- Base quantization tag for comparison (default: fp16, or uses existing base from results file)-D <url>- Remote server URL (default: local)-O <file>- Output results file (default: modelname.qc.json)-Te <value>- Temperature (default: 0.0 for deterministic results)-S <value>- Random seed (default: 365)-To <value>- top_p parameter (default: 0.001)-Top <value>- top_k parameter (default: -1)-R <value>- repeat_penalty parameter-Fr <value>- frequency_penalty parameter-T <file>- External test suite JSON file (default: internal v1base)--force- Force re-run testing for quantizations already present in results file--rejudge- Re-run judgment process for existing test results (without re-testing)--judge <model>- Use a judge model for similarity scoring (see Judge Scoring below)--judge-ctxsize <value>- Context length for judge model (0 = auto, default: 0). Auto mode calculates: test_ctx × 2 + 2048--mode <mode>- Judge execution mode:serial(default) orparallel--timeout <seconds>- API timeout in seconds for testing and judgment calls (default: 600)--verbose- Show judgment details (question ID, score, reason) for each judged question--ondemand- Pull models on-demand if not available, then remove after testing (see On-Demand Mode below)--repo <url>- Repository URL for the model source (saved in results file for qcview)--overwrite- Overwrite existing output file without prompting--enablethinking- Enable thinking mode for thinking models (disabled by default)--thinklevel <level>- Set thinking level (low, medium, high) - overrides --enablethinking--no-unloadall- Skip unloading all models before testing--logfile <path>- Log process output to file (appends if exists, timestamps each line, strips color codes)--fix- Attempt to fix a corrupted/malformed results file and recover data (outputs to .fixed.json)
Judge Scoring:
Use a second LLM to evaluate similarity between base and quantized responses:
# Local judge model
osync qc -M llama3.2 -Q 1b-instruct-q8_0,1b-instruct-q6_k --judge mistral
# Remote test models, local judge
osync qc -d http://192.168.1.100:11434/ -M llama3.2 -Q q4_k_m --judge mistral
# Remote judge model (different server)
osync qc -M llama3.2 -Q q4_k_m --judge http://192.168.1.200:11434/mistral
# Parallel mode (judge runs concurrently with testing)
osync qc -M llama3.2 -Q q4_k_m --judge mistral --mode parallel
# Re-run judgment only (use existing test results)
osync qc -M llama3.2 -Q q4_k_m --judge mistral --rejudge
When judgment scoring is enabled:
- The judge model evaluates each quantized response against the base response
- Returns a similarity score from 1-100 for each question with reasoning
- Final score is calculated as: 50% metrics + 50% judgment
- Results display shows separate columns for Final Score, Metrics Score, and Judge Score
- Existing judgments are skipped unless
--force,--rejudgeis used, or a different judge model is specified
Separate Best Answer Judge Model:
Use --judgebest to specify a different model for best answer determination (A/B/tie judgment):
# Same model for similarity and best answer (when using --judge alone)
osync qc -M llama3.2 -Q q4_k_m --judge mistral
# Different model for best answer judgment
osync qc -M llama3.2 -Q q4_k_m --judge mistral --judgebest llama3.3
# Best answer judgment only (no similarity scoring)
osync qc -M llama3.2 -Q q4_k_m --judgebest llama3.3
# Remote best answer judge
osync qc -M llama3.2 -Q q4_k_m --judge mistral --judgebest http://192.168.1.200:11434/llama3.3
# Re-run only best answer judgment with new model
osync qc -M llama3.2 -Q q4_k_m --judgebest llama3.3 --rejudge
When --judgebest is specified:
- Per quantization: similarity judgment runs first (if
--judgeis used), then best answer judgment - Best answer model uses a quality-focused prompt (no similarity scoring)
- Results store
JudgeModelBestAnswer,ReasonBestAnswer, andJudgedBestAnswerAttimestamps - QcView outputs show both judge models when different
Cloud Provider Support:
Use cloud AI providers (Anthropic Claude, OpenAI, etc.) as judge models with the @provider/model syntax:
# Using environment variable for API key
osync qc -M llama3.2 -Q q4_k_m --judge @claude/claude-sonnet-4-20250514
osync qc -M llama3.2 -Q q4_k_m --judge @openai/gpt-4o
# Using explicit API key
osync qc -M llama3.2 -Q q4_k_m --judge @claude:sk-ant-xxx/claude-sonnet-4
# Azure OpenAI (key@endpoint format)
osync qc -M llama3.2 -Q q4_k_m --judge @azure:mykey@myendpoint.openai.azure.com/gpt4-deployment
# Mixed: cloud judge with ollama best answer judge
osync qc -M llama3.2 -Q q4_k_m --judge @claude/claude-sonnet-4 --judgebest llama3.2:latest
# Both judges from cloud providers
osync qc -M llama3.2 -Q q4_k_m --judge @openai/gpt-4o --judgebest @claude/claude-opus-4
Supported cloud providers and their environment variables:
| Provider | Syntax | Environment Variable |
|---|---|---|
| Anthropic Claude | @claude/model | ANTHROPIC_API_KEY |
| OpenAI | @openai/model | OPENAI_API_KEY |
| Google Gemini | @gemini/model | GEMINI_API_KEY or GOOGLE_API_KEY |
| Azure OpenAI | @azure/deployment | AZURE_OPENAI_API_KEY + AZURE_OPENAI_ENDPOINT |
| Mistral AI | @mistral/model | MISTRAL_API_KEY |
| Cohere | @cohere/model | CO_API_KEY or COHERE_API_KEY |
| Together AI | @together/model | TOGETHER_API_KEY |
| HuggingFace | @huggingface/model | HF_TOKEN or HUGGINGFACE_TOKEN |
| Replicate | @replicate/model | REPLICATE_API_TOKEN |
Use osync qc --help-cloud for detailed provider documentation and examples.
When using cloud providers:
- API keys can be provided via environment variables (default) or explicitly in the command line
- Connection and model validation is performed before testing starts
- Cloud provider info (name and API version) is recorded in results for traceability
- QcView displays cloud provider badges in HTML/PDF output (only for cloud providers, not for Ollama)
Judge Execution Modes:
-
Serial mode (default): After testing each quantization, all questions are judged sequentially before moving to the next quantization. Simple and predictable execution.
-
Parallel mode: Testing and judgment run concurrently at the question level. As each question is tested, it is immediately passed to the judge model in a background task. This allows the test model to continue generating answers while the judge evaluates previous questions. Significantly reduces total execution time when using a remote judge or when the judge model is faster than the test model.
On-Demand Mode:
Use --ondemand to automatically pull models that aren't available and remove them after testing. This is ideal for testing large models or many quantizations without consuming permanent storage:
# Test many quantizations without keeping them all
osync qc -M llama3.2 -Q q2_k,q3_k_s,q3_k_m,q4_k_s,q4_k_m,q5_k_s,q5_k_m,q6_k,q8_0 --ondemand
# Test on remote server with on-demand
osync qc -M llama3.2 -Q q4_k_m,q8_0 -D http://192.168.1.100:11434 --ondemand
When on-demand mode is enabled:
- Before testing each quantization, osync checks if the model exists on the Ollama instance
- If the model is missing, it is automatically pulled from the registry
- Models that were already present are NOT removed after testing (only on-demand pulled models)
- After testing and judgment complete, on-demand pulled models are removed to free disk space
- The on-demand state is saved in the results file, so interrupted tests can properly clean up on resume
- Works with both local and remote Ollama servers
Built-in Test Suites:
v1base(default) - 50 questions across Reasoning, Math, Finance, Technology, Sciencev1quick- 10 questions (subset of v1base for quick testing)v1code- 50 coding questions across Python, C++, C#, TypeScript, Rust (8192 max tokens)
External Test Suite Format:
Create custom test suites using JSON files with the following structure:
{
"name": "my-custom-suite",
"numPredict": 4096,
"contextLength": 4096,
"categories": [
{
"id": 1,
"name": "Category Name",
"contextLength": 8192,
"questions": [
{
"categoryId": 1,
"questionId": 1,
"text": "Your question text here",
"contextLength": 16384
}
]
}
]
}
numPredict- Maximum tokens generated per response (default: 4096). Use higher values for coding or detailed answers.contextLength- Context length (num_ctx) for the model during testing. Can be specified at:- Suite level (default: 4096) - applies to all questions
- Category level - overrides suite default for all questions in that category
- Question level - overrides both suite and category settings for that specific question
Use with: osync qc -M model -Q q4_k_m -T my-custom-suite.json
Reference files v1base.json and v1code.json are included in the osync directory.
The recommended model to use as a judge is gemma3:12b, it is advisable to use a large context size (6k for v1base and 12k for v1code).
It's also recommended to run the qc command with the --verbose argument to verify the scoring and reason given corresponds to the expected beahviour:
Judging UD-IQ3_XXS Q1-1 Score: 92% (1/50 2%)
A and B match: Both responses provide a complete, thread-safe LRU cache implementation in Python using `OrderedDict` and `threading.RLock`. They both include comprehensive docstrings, error
handling, and a full set of methods (get, put, delete, clear, size, etc.). The core logic for managing the cache, including eviction and thread safety, is nearly identical. The primary differences
are in the formatting of the docstrings and some minor wording variations in the explanations. Both responses also include example usage and testing code. The code itself is very similar, with
only minor differences in variable names and comments. Overall, the responses demonstrate a very high degree of similarity in terms of content, approach, and functionality.
Judging UD-IQ3_XXS Q1-2 Score: 75% (2/50 4%)
A and B match: Both responses provide a Python async web scraper using aiohttp, incorporating concurrent crawling, rate limiting, retries, and CSS selector-based data extraction. They both utilize
asyncio, aiohttp, logging, and dataclasses. Both include comprehensive error handling and logging. However, they differ in their implementation details. Response A uses a semaphore for concurrency
control and a class-based structure for the scraper, while Response B introduces a separate RateLimiter class and a more streamlined approach to data extraction. Response A's retry logic is more
detailed, including random jitter, while Response B's is simpler. Response B also includes an advanced scraper with custom selectors.
Judging UD-IQ3_XXS Q1-3 Score: 75% (3/50 6%)
A and B differ: Both responses implement a retry decorator factory with similar functionality (configurable max attempts, delay strategies, exception filtering, and support for both sync and async
functions). However, they differ significantly in their implementation details and structure. Response A uses a class-based approach with a `RetryError` exception and separate `_calculate_delay`
function. It also includes convenience decorators for common retry patterns. Response B uses a dataclass for configuration and an Enum for delay strategies, making it more structured. It also has
separate functions for sync and async decorators. While both achieve the same goal, the code organization and specific techniques used are quite different, leading to a noticeable difference in
View Quantization Results (qcview)
Display quantization comparison results in formatted tables or export to various formats.
# View results in table format (console)
osync qcview llama3.2.qc.json
# Export table to text file
osync qcview llama3.2.qc.json -O report.txt
# Export as JSON to file
osync qcview llama3.2.qc.json -Fo json -O report.json
# Export as Markdown
osync qcview llama3.2.qc.json -Fo md
# Export as HTML (interactive with theme toggle)
osync qcview llama3.2.qc.json -Fo html
# Export as PDF (includes full Q&A pages)
osync qcview llama3.2.qc.json -Fo pdf
# Add or override repository URL in output
osync qcview llama3.2.qc.json -Fo html --repo https://github.com/user/model
Table Output:
Displays color-coded results with:
- Overall confidence scores (0-100%) for each quantization
- Component scores: Token Similarity, Logprobs Divergence, Length Consistency, Perplexity
- Category-by-category breakdown
- Performance metrics (eval/prompt tokens per second)
- Model metadata (quantization type with tensor analysis, disk size)
- Shows dominant tensor quantization with percentage (e.g.,
Q4_K (87%)) - Mixed quantization shown as
Q6_K (81% Q8_0)when dominant differs from tag - Unknown tensor types indicated with
?suffix (e.g.,Q3_K?)
- Shows dominant tensor quantization with percentage (e.g.,
Color Coding:
- 🟢 Green (90-100%): Excellent quality preservation
- 🟢 Lime (80-90%): Very good quality
- 🟡 Yellow (70-80%): Good quality
- 🟠 Orange (50-70%): Moderate quality loss
- 🔴 Red (<50%): Significant quality degradation
Output Formats:
- table (default) - Color-coded console output or plain text file
- json - Complete results with all scores and metadata
- md - Markdown format with tables (for GitHub, documentation)
- html - Interactive HTML with dark/light theme toggle, collapsible Q&A sections
- pdf - Professional PDF report with:
- Summary tables with color-coded scores
- Category breakdown table
- Rankings by score, speed, perplexity, and best answers
- Full Q&A pages for each quantization (questions, answers, judgment details)
Options:
<file>- Results file to view (required, positional argument)-Fo <format>- Output format: table, json, md, html, pdf (default: table)-O <file>- Output filename (default: auto-generated based on format)--repo <url>- Repository URL (displayed in output, overrides value from results file)--metricsonly- Ignore judgment data and show only metrics-based scores (useful for comparing pure model output quality without judge influence)--overwrite- Overwrite existing output file without prompting
Context Benchmark (bench)
Run context tracking benchmarks to evaluate how well models maintain information across long conversations.
# Basic benchmark with a model
osync bench -M llama3.2
# Test on remote server (multiple ways to specify)
osync bench -M llama3.2 -D http://192.168.1.100:11434
osync bench -M llama3.2 -D 192.168.1.100 # IP (auto: http:// + :11434)
osync bench -M llama3.2 -D myserver/ # trailing slash
# With judge evaluation
osync bench -M llama3.2 --judge gemma3:12b
# Custom output file
osync bench -M llama3.2 -O my-benchmark.json
# Enable thinking for thinking models (qwen3, deepseek-r1) - disabled by default
osync bench -M qwen3 --enablethinking
# Control thinking level (overrides --enablethinking)
osync bench -M qwen3 --thinklevel medium
# Skip unloading all models before testing
osync bench -M llama3.2 --no-unloadall
# Generate a custom test suite
osync bench --generate-suite -T custom-suite.json -O custom-suite.json
How It Works:
The bench command generates dynamic stories with embedded facts and tests the model's ability to:
- Answer questions about facts in the current story category (New questions)
- Recall facts from previous categories in the conversation (Old questions)
- Execute tool calls when tool mode is enabled
Question Types:
- New - Questions about facts introduced in the current category (tests comprehension)
- Old - Questions requiring retrieval of facts from earlier in the conversation (tests context tracking)
- Tool - Questions requiring tool/function calls (when tools are enabled)
Scoring:
Each question is evaluated as:
- Correct - Answer contains the expected information
- Incorrect - Answer is wrong or missing key information
- Partial - Answer is partially correct (when applicable)
Options:
-M <name>- Model name without tag (required unless --help-cloud, --showtools, or --generate-suite)-Q <tags>- Model/Quantization tags to compare (comma-separated, supports wildcards e.g., Q4*,IQ*)-D <url>- Remote server URL (default: local)-O <file>- Results output file (default: modelname.testtype.json)-T <file>- Test suite file or type. For bench: filename first, falls back to v1.json if type given. For --generate-suite: test type (ctxbench/ctxtoolsbench)-L <category>- Limit testing to a specific category (e.g., 2k, 4k, 8k, 16k, 32k, 64k, 128k, 256k)-Te <value>- Model temperature (default: from model template)-S <value>- Model seed (default: 365)-To <value>- Model top_p (default: from model template)-Top <value>- Model top_k (default: from model template)-R <value>- Model repeat_penalty (default: from model template)-Fr <value>- Model frequency_penalty (default: from model template)--force- Force re-run testing for quantizations/models already present in results file--rejudge- Re-run judgment process for existing test results (without re-testing)--judge <model>- Judge model for answer evaluation. Ollama: model_name or http://host:port/model. Cloud: @provider[:key]/model--mode <mode>- Judge execution mode:serial(default) orparallel--timeout <seconds>- API timeout in seconds (default: 1800)--verbose- Show testing conversation, tools usage, judgment details--judge-ctxsize <value>- Context length for judge model (0 = auto based on test type)--ondemand- Pull models on-demand if not available, then remove after testing--repo <url>- Repository URL for the model source (saved per model/quant in results file)--showtools- Display available tools with descriptions and queryable data--generate-suite- Generate test suites. Use with -T (ctxbench/ctxtoolsbench) for specific type, -O for custom filename--help-cloud- Show detailed help for cloud provider integration--enablethinking- Enable thinking mode for thinking models (disabled by default)--thinklevel <level>- Set thinking level (low, medium, high) - overrides --enablethinking--no-unloadall- Skip unloading all models before testing--overwrite- Overwrite existing output file without prompting--ctxpct <value>- Content scaling percentage for test suite generation (default: 100, range: 50-150)--logfile <path>- Log process output to file (appends if exists, timestamps each line, strips color codes)--calibrate- Enable token calibration mode: detailed tracking of estimated vs actual tokens at each step--calibrate-output <file>- Output file for calibration data (default: calibration_.json in test suite directory)--fix- Fix corrupted/malformed results JSON file (specify file with -O or -M)
Bench Test Suite Format:
Custom bench test suites can specify a numPredict field to control maximum tokens per response:
{
"testType": "ctxbench",
"testDescription": "Custom context benchmark",
"numPredict": 16384,
"maxContextLength": 131072,
"categories": [...]
}
numPredict- Maximum tokens generated per test response (default: 16384)maxContextLength- Maximum context length supported by the test suite- Judge responses use a fixed 8192 token limit
View Context Benchmark Results (benchview)
Display context benchmark results in formatted tables or export to various formats.
# View results in table format (console)
osync benchview llama3.2.ctxbench.json
# Export to different formats
osync benchview results.json -Fo json -O report.json
osync benchview results.json -Fo md -O report.md
osync benchview results.json -Fo html -O report.html
osync benchview results.json -Fo pdf -O report.pdf
Output Information:
- Overall accuracy percentage
- Category-by-category breakdown
- Question type analysis (New vs Old accuracy)
- Tool usage statistics (when applicable)
- Detailed Q&A with model responses
Output Formats:
- table (default) - Color-coded console output
- json - Complete results with all scores and responses
- md - Markdown format with tables
- html - Interactive HTML with theme toggle
- pdf - Professional PDF report
Options:
<file>- Results file to view (required, positional argument)-Fo <format>- Output format: table, json, md, html, pdf (default: table)-O <file>- Output filename (default: auto-generated based on format)-C <category>- Filter to specific category--details- Show detailed Q&A results in output--overwrite- Overwrite existing output file without prompting
Manage (manage)
Interactive TUI for managing models with keyboard shortcuts.
# Launch manage interface for local server
osync manage
# Launch manage interface for remote server (multiple ways to specify)
osync manage http://192.168.0.100:11434
osync manage 192.168.0.100 # IP (auto: http:// + :11434)
osync manage myserver/ # trailing slash
Features:
- Full-screen terminal user interface (Terminal.Gui 2), lists the models through the server's API
- Column colors (name, size, parameters, quantization, family, age, ID),
●marks models loaded in memory - Multi-selection for batch copy, delete and update
- Filter by typing, multiple sort modes
- 34 themes with live preview; the chosen theme is saved in the preferences file
- Settings dialog: local server (Ollama / xOllama, host, port) and color mode, saved in the preferences file
- Copy, run, update and pull run on the plain console and return to the list, with the cursor on the model
- On an xOllama server: the model details list the model's xOllama settings, and Ctrl+W changes them with
xollama tweak
Keys:
- Up/Down, PgUp/PgDn, Home/End - Move
- Space - Select / unselect the model (and move down)
- Enter or Right - Model details
- Letters, digits,
: - _ . /- Filter by name (*matches any text); Backspace removes a character - Esc - Clear the filter, or exit (with confirmation)
- F1 - Help
- F2 - Rename model (also Ctrl+M where the terminal tells it apart from Enter)
- Ctrl+Left / Ctrl+Right - Previous / next server: the local one, the other of Ollama/xOllama side by side, and the aliases chosen with
osync setup manage servers(also asked byosync setup server) - Ctrl+C - Copy model(s) (on the same server, or to a remote server; batch with selected models)
- Ctrl+R - Run/chat with model
- Ctrl+S - Show license, Modelfile, parameters, system prompt or template
- Ctrl+D - Delete model(s) (Enter in the confirmation cancels)
- Ctrl+U - Update model(s)
- Ctrl+P - Pull model from a registry (Ollama library names are checked before leaving the list)
- Ctrl+L - Load model into memory (in the background)
- Ctrl+K - Unload model from memory
- Ctrl+X - Loaded models (ps)
- Ctrl+O - Cycle sort order
- Ctrl+T - Choose the theme (live preview, saved)
- Ctrl+E - Settings: local server and color mode (saved)
- Ctrl+W - Tweak (xOllama servers): change the model's xOllama settings (see below)
- Ctrl+Q - Quit
Tweak (xOllama): xOllama keeps settings of its own in the model: engine, KV cache types, dynamic slots, DCA, session affinity and prefix pooling, council, GPU/devices, speculative decoding. They are listed in the model details (Enter), and on an xOllama server Ctrl+W changes them for the model under the cursor, or for each selected model. Pick what to change (every setting, or one group such as KV cache or council), or remove the settings. You can also type flags: a flag with a value is set without questions, e.g. --kv-k=q8_0 --kv-v=q8_0 or --council=on.
osync runs xollama tweak model <model> on the console against the server manage shows, including a remote one. The questions, the explanations and the consistency checks are xOllama's own, and so is the list of settings, so it always matches the xOllama version. This needs the xollama CLI on this machine: on PATH, or set OSYNC_XOLLAMA_CLI to its path. Only the xOllama settings layer changes: weights, template, system prompt and parameters stay as they are. See xOllama's tweak documentation.
Sort Modes:
- Name+ (ascending), Name- (descending)
- Size+ (ascending), Size- (descending)
- Created+ (oldest first), Created- (newest first)
Themes (osync setup manage themes shows them with a preview):
- Dark: Default, Dracula, Nord, Tokyo Night, Catppuccin Mocha, Gruvbox Dark, Monokai, Solarized Dark, Ocean, Matrix, High Contrast, Classic, One Dark, GitHub Dark, Everforest, Rose Pine, Kanagawa, Ayu Mirage, Night Owl, Material Ocean, Synthwave, Palenight, Iceberg, Zenburn, Cobalt, Horizon, Amber
- Light: Solarized Light, Light, Catppuccin Latte, Gruvbox Light, One Light, Ayu Light, Rose Pine Dawn
Setup (setup)
Preferences in the settings file, by section. Without the last arguments a section asks interactively (numbered choices, Enter keeps the current value).
osync setup # summary, then a menu
osync setup show # summary only
osync setup server # ask: Ollama, xOllama or both, host, ports
osync setup server xollama 192.168.1.5 # one server (port: the flavor's default)
osync setup server ollama nas:11500
osync setup server both [host] [xollama] # both side by side (default ports; optional default server)
osync setup server auto # back to auto-detection
osync setup server env ignore # the settings win over XOLLAMA_HOST / OLLAMA_HOST (env use: they win)
osync setup alias # list (and add/remove interactively)
osync setup alias add gpu 192.168.1.10 # port 11434, or 22434 when only xOllama answers
osync setup alias remove gpu
osync setup manage # theme and default sort order
osync setup manage theme "Tokyo Night" # name, loose spelling (tokyo-night) or number
osync setup manage sort size- # name+ name- size- size+ created- created+
osync setup manage servers gpu,nas # aliases for Ctrl+Left/Right in manage (all, none)
osync setup manage themes # all themes with a preview
osync setup shell # theme, color mode, tab completion
osync setup shell theme gruvbox-light # or: plain, default
osync setup shell colors truecolor # auto, truecolor, 256, 16, none
osync setup shell completion # install tab completion (bash / PowerShell)
osync setup shell themes
Version (showversion, -v)
Display osync version and environment information.
# Show version only
osync -v
osync showversion
# Show detailed environment information
osync showversion --verbose
Basic Output:
osync v1.2.6 (b20260110-1117)
Verbose Output:
osync v1.2.6 (b20260110-1117)
Binary path: C:\Users\user\.osync\osync.exe
Installed: Yes (C:\Users\user\.osync)
Shell: PowerShell Core 7.5.4
Tab completion: Installed (C:\Users\user\Documents\PowerShell\Microsoft.PowerShell_profile.ps1)
Verbose Information:
- Binary path - Location of the osync executable
- Installed - Whether osync is installed in the standard install directory
- Shell - Detected shell type and version (bash, zsh, PowerShell Core/Desktop, cmd)
- Tab completion - Status of tab completion for the current shell
Install (install)
Install osync to user directory and configure shell completion.
# Run the installer
osync install
Installation Directory:
- Windows:
~/.osync - Linux/macOS:
~/.local/bin
Features:
- Copies osync executable to install directory
- Adds install directory to PATH
- Offers to configure tab completion for detected shell
- Supports bash, zsh, and PowerShell
Options
Global Options
-h,-?- Show help for any command-bt <value>- Bandwidth throttling (B, KB, MB, GB per second)-BufferSize <value>- Memory buffer size for remote-to-remote copy (KB, MB, GB; default: 512MB)
Examples:
# Bandwidth throttling
osync cp llama3 http://server:11434 -bt 75MB # Limit to 75 MB/s
osync cp qwen2 http://server:11434 -bt 1GB # Limit to 1 GB/s
# Memory buffer configuration for remote-to-remote
osync cp http://server1:11434/llama3 http://server2:11434/llama3 -BufferSize 256MB
osync cp http://server1:11434/qwen2 http://server2:11434/qwen2 -BufferSize 1GB
List Options
--size- Sort by size (largest first)--sizeasc- Sort by size (smallest first)--time- Sort by modified time (newest first)--timeasc- Sort by modified time (oldest first)
Interactive Mode
Run osync without arguments to enter interactive mode with:
- Tab completion for model names
- Command history
- Multi-registry support
- Real-time model listing
osync
# Press tab to see available models
# Type command and press enter
> cp llama3 my-backup
> ls "qwen*"
> quit # or type 'exit', or press Ctrl+C
Pattern Matching
Use * wildcard for flexible pattern matching:
# Match any model starting with "llama"
osync ls "llama*"
# Match any 7b model
osync ls "*:7b"
# Match models in namespace
osync ls "mannix/*"
# Match HuggingFace models
osync ls "hf.co/*"
# Match any model containing "test"
osync ls "*test*"
Examples
Update Models
# Update all local models to latest versions
osync update
# Update specific models matching pattern
osync update "llama*"
# Update all models on remote server
osync update http://192.168.0.10:11434
Backup Before Update
# Create backup before updating
osync cp llama3:latest llama3:backup
osync update llama3
Deploy to Multiple Servers
# Upload from local to multiple servers (short form with IP)
osync cp llama3 192.168.0.10
osync cp llama3 192.168.0.11
osync cp llama3 192.168.0.12
# Or using hostnames with trailing slash
osync cp llama3 server1/
osync cp llama3 server2/
osync cp llama3 server3/
# Copy between remote servers
osync cp http://192.168.0.10:11434/llama3 http://192.168.0.11:11434/llama3
osync cp http://192.168.0.10:11434/qwen2 http://192.168.0.12:11434/qwen2
Clean Up Old Models
# List and remove test models
osync ls "test-*"
osync rm "test-*"
# Remove old backups
osync rm "*:backup"
Organize Models
# List all models by size to find space hogs
osync ls --size
# Rename for better organization
osync mv llama3 llama3-8b:prod
osync mv qwen2 qwen2-7b:dev
Known Issues
None
Changelog
v1.4.2
New
managetweaks xOllama model settings - on an xOllama server, Ctrl+W runsxollama tweak modelfor the model under the cursor, or for each selected model, against the server manage shows. You can walk every setting, one group (KV cache, dynamic slots, DCA, session pooling, council, GPU/devices, engine), or remove the settings, and add flags such as--kv-k=q8_0that are set without questions. The model details list the xOllama settings. Needs thexollamaCLI on PATH, orOSYNC_XOLLAMA_CLI
v1.4.1
Fixes
- Copies between servers are byte-for-byte again when the destination cannot pull from the relay (for example a Windows destination: the relay's
host:portis not a valid folder name there). The model used to be recreated from/api/show, which dropped the renderer, parser,requiresand xOllama's model settings (a council became a plain model), merged several licenses and changed the parameters, yet osync reported success. It is now recreated from the source's manifest with the config and settings layers verbatim, so every layer and the config have the source's digests - Uploads of a local model to a remote server keep every part of the model the same way (they parsed
ollama show --modelfileand lost the same parts) - The relay no longer tries port 80 first: it listens on a free port the OS assigns (port 80 is often taken, and needs root on Linux/macOS). Port 80 is used only for a Windows source server, which needs a relay name without a port, and only if it is free
HF_TOKENis sent on osync's own HuggingFace requests (tag lookups forhf.co/...wildcard tags,qcmodel discovery), for higher rate limits and private or gated repositories; only to huggingface.co / hf.co over HTTPS- Every copy between servers is verified: osync compares the copy's modelfile with the source's and fails, instead of printing "Successfully copied", when a part differs; the final line says how the model was installed (
through the relay,recreated from its manifest)
v1.4.0
New
- xOllama support - osync works with the xOllama fork as well as Ollama
- Local server discovery:
XOLLAMA_HOST,OLLAMA_HOST, thenlocalhost:11434/localhost:22434 - Server flavor detection (
osync psshowsServer: Ollama|xOllama at <url>) - Local operations use the
xollamaCLI when the local server is xOllama (override withOSYNC_OLLAMA_CLI) XOLLAMA_MODELShonored for the models directory; xOllama processes shown in process stats
- Local server discovery:
- Ollama and xOllama side by side - server type "Both": one is the default local server, each gets an alias (
ollama,xollama) - Remote-to-remote and remote-to-local copy of any model - new push relay: the source server pushes the model to a temporary registry endpoint run by osync, which streams the blobs straight into the destination. Models that only exist on the source server (created, imported, HuggingFace) can now be copied, the complete manifest is preserved (license, messages, projector, ...), copies work between Ollama and xOllama, and no internet access is needed (previously blobs were downloaded from registry.ollama.ai)
osync setup- one command for the preferences:server(Ollama, xOllama or both),alias(server aliases),manage(theme, default sort order, servers),shell(colors, theme, tab completion),show; interactive or with arguments for scripts- Preferences file -
settings.jsonin the per-OS configuration folder: local server (Ollama/xOllama, host, port), aliases, color mode,manageand shell settings - Server aliases -
osync setup alias add gpu 192.168.1.10, then-d gpu,osync cp model gpu/,osync cp gpu/model copy,osync manage gpu - The settings can win over
XOLLAMA_HOST/OLLAMA_HOST-osync setup serverasks when one is set,osync setup server env ignore|use, and a check box in the manage settings (Ctrl+E) managerewritten on Terminal.Gui 2 - true color with multi-color themes (one color per column,●for models loaded in memory, colored top and bottom bars) adapted to 256 and 16-color terminals with contrast checks; theme picker with live preview (Ctrl+T), the theme is saved in the preferences file; settings dialog (Ctrl+E) for the local server (Ollama/xOllama, host, port, connection test) and the color mode; column headers; F1 help; rename on F2 (Ctrl+M is Enter in most terminals); load runs in the background; console operations (copy, run, update, pull) return to the list without restarting osync; pull validation no longer rejectshf.co/...models; confirmations default to the safe answer (Enter cancels a delete)manageswitches servers with Ctrl+Left / Ctrl+Right - the local server, the other one of Ollama/xOllama side by side, and the aliases chosen withosync setup manage servers(osync setup serveroffers them); the top bar shows which one ([2/3] gpu: Ollama @ ...)- Colored command output again -
ls,ps,show,-v, copy/pull/update progress, errors, warnings and results use the shell theme (removed in 1.0.1 because of garbled output on Linux); plain when redirected, withNO_COLORor theplaintheme - 34 themes for
manageand the shell, 7 of them for light terminals, all checked for readable contrast at true color, 256 and 16 colors - Color depth detection - true color / 256 / 16 colors detected from the real terminal (it was read after osync had replaced
TERMwithxterm-16color, so every terminal was treated as 16-color);colorModesetting,OSYNC_COLOR_MODEandNO_COLORoverride it;osync -v --verboseshows the result osync installasks for the local server only when needed - a single server found on this machine is used without questions; when none or both are found it asks for the type (Ollama, xOllama or both), host and port, with detected defaults and a connection test; installing from a renamed binary (e.g.osync-macos-arm64 install) works- Build timestamp embedded at compile time -
osync -vshows the real build time (UTC) for every binary, including renamed ones (osync-macos-arm64) and downloaded copies, instead of the file's modification time
Fixes
- Fixed
osync <command> -hand missing-argument errors hanging forever on Linux/macOS when output is redirected (pipes, scripts, CI): the help renderer looped endlessly at 100% CPU with growing memory - Fixed
run,ps,qcignoringOLLAMA_HOST;manage,psmonitorand local judge models now use the same local-server resolution - Fixed
pstruncating model names to 20 characters when output is redirected - Fixed remote
showprinting only the Modelfile: it now shows the same sections asollama show(model details, capabilities, projector, parameters, system, license; all metadata with-v), several section flags can be combined, and a missing model exits with an error - Fixed
rmandupdateexiting with code 0 when no model matches, or when deleting/updating a model failed (updateof all models on an empty server is still a success) - Fixed server URLs without a port using port 80 when the model has a tag (e.g.
osync cp http://server/qwen3:4b ...): the:of the tag was taken for a port. A server given without a port now uses 11434, or 22434 when the host refuses 11434 but accepts 22434 (xOllama) - Fixed
manageshowingunknownquantization (and no parameters/family) when the local models directory and the resolved local server did not match (e.g. Ollama and xOllama both installed): local models are now listed through the local server's API, so the list and its details always come from the same server, and startup no longer makes one/api/showcall per model. The top bar shows which server is used (Ollama @ localhost:11434); the models directory is only read when the server is unreachable, which the top bar says - Fixed copy to a remote server failing when the local models directory is not readable (e.g. the server runs as the
ollamaservice user): the upload now goes through the local server with the push relay - Fixed the theme list of
managenot scrolling when the terminal is shorter than the list - Fixed bandwidth throttling (
-bt) not limiting short bursts, counting requested instead of read bytes, and misbehaving after ~25 days of uptime - No stray ANSI reset (
ESC[0m) in redirected output on Linux/macOS
Platform
- .NET 10 - Retargeted from .NET 8 (Windows-only target framework) to cross-platform
net10.0; builds and tests on Windows, Linux and macOS - macOS binaries - Releases now include
osync-macos-arm64andosync-macos-x64 - New test suite and CI - Atomic Reqnroll integration scenarios run against real Ollama and xOllama servers (local + two remotes) on every push;
devbuilds publish pre-releases,masterpublishes releases
v1.2.9
- Real-Time Monitor Improvements - Enhanced monitoring dashboard
- Added
-Hishortcut for--historyargument (e.g.,osync monitor -Hi 10) - Plain integers in
--historyare now treated as minutes (e.g.,-Hi 10= 10 minutes) - Moved date/time from header to status bar (right-justified), reclaiming one line of vertical space
- Status bar now always pinned to bottom of terminal window
- Added osync version with build number in Ollama panel (e.g.,
osync v1.2.9 (b20260116-1814)) - Reduced screen flicker during refresh by overwriting content in place
- Fixed display artifacts when models load/unload (consistent table structure)
- Fixed graph scaling to properly fill the graph width based on history duration
- Time axis shows seconds when history < 5 minutes (avoids repeated minute labels)
- Fixed model expiration time calculation (was showing wrong time due to timezone handling)
- Ollama process metrics now aggregate ALL related processes (ollama serve, runners, llama-server)
- Shows process count in Ollama panel when multiple processes exist (e.g., "28 procs, 3.9 GB")
- Dynamic graph headers: Shows "Ollama" (blue/orange) when per-process data available, "System" (yellow/steel) when using system-wide fallback
- VRAM fallback: Uses Ollama API SizeVram when nvidia-smi can't report per-process VRAM (Windows/WDDM)
- GPU utilization fallback: Uses total GPU utilization when per-process utilization unavailable
- NvAPIWrapper integration (Windows only): Additional NVIDIA GPU data source as fallback
- D3DKMT per-process GPU monitoring (Windows only): Native Windows kernel API for accurate per-process GPU utilization and VRAM tracking, works with any GPU vendor (NVIDIA, AMD, Intel)
- Improved process name matching for Ollama detection (supports ollama_llama_server, llama-server, runner)
- Added
- New Bench Command - Context tracking benchmark with dynamic story-based tests
- Generates stories with embedded facts across multiple categories
- Tests model's ability to track context through conversation
- Question types: New (current category facts), Old (retrieval from previous categories)
- Tool calling support for function execution tests
- Judge evaluation for answer quality assessment
--enablethinkingand--thinklevelarguments for thinking models (qwen3, deepseek-r1)--no-unloadallargument to skip unloading all models before testing--overwriteargument to skip file overwrite prompts--generate-suiteto create custom test suite JSON files with-Tand-Ooptions- Configurable context length, temperature, seed, and other parameters
- Progress bar during testing with timing statistics (Last/Avg/Max response times)
- Improved pull progress display with download speed and ETA
- Thinking token tracking: separate tracking for model thinking/reasoning tokens with verbose output
- Character consistency: story generator maintains consistent name-to-animal mapping across all chapters
- Optimized message flow: instructions and context combined with first question (avoids model confusion)
- Context length management: auto-detects model max context, configurable overhead (2K normal, 4K thinking)
- Two-phase HuggingFace rate limit retry: 50 quick retries (2s delay), then 50 slow retries (30s/API delay)
- Auto-backup: creates .backup.zip of existing results file before continuing (protects against data loss)
--mode=parallelfor parallel judgment - judges answers in background while testing continues- Parallel mode dual progress bars: test progress bar shows test model metrics (Avg/Max time, p:/e: tok/s), judge progress bar shows judge model metrics separately
- Pre-flight check caching: thinking detection, context settings, and tools validation cached per model (SHA256 verified)
- New BenchView Command - View and export context benchmark results
- Multiple output formats: table (console), json, md (markdown), html, pdf
- Category breakdown with accuracy percentages
- Question type analysis (New vs Old fact retrieval)
- Tool usage statistics when applicable
--overwriteargument to skip file overwrite prompts
- Increased num_predict Limits - Larger token limits for improved response quality
- Bench pre-flight check: 512 → 2048 (fixes issues with some models)
- Bench test responses: 2048 → 16384 (configurable via test suite
numPredict) - Bench judge: added 8192 limit (was missing)
- QC judge: 800 → 8192 (reduces truncated judge responses)
- Bench Test Suite Configuration - New
numPredictfield in bench test suite JSON- Controls maximum tokens generated per test response
- Default: 16384 tokens
- Can be customized per test suite for different use cases
- Enhanced Process Status (
ps) - Extended system monitoring- Shows Ollama process CPU and memory usage when running locally
- GPU monitoring for NVIDIA cards (uses nvidia-smi): utilization, memory, temperature, power
- GPU monitoring for AMD cards (uses rocm-smi): utilization, memory, temperature, power
- Ollama-specific VRAM usage per GPU (shows percentage of total GPU memory)
- Color-coded output: green (0-50%), yellow (50-75%), orange (75-90%), red (90-100%)
- Temperature color coding: green (<60°C), yellow (60-70°C), orange (70-80°C), red (>80°C)
- Automatically detects available GPU monitoring tools
- Load/Unload Command Improvements
- Load command now shows proper "Model not found" error instead of misleading connection error
- Both commands verify status using /api/ps after operation completes
- Better error messages for different HTTP status codes (404, 500, etc.)
- CLI Improvements
- Shortened
osync -houtput to show only global options and available commands - Use
osync <command> -hfor detailed help on specific commands - Fixed ANSI color bleed on Linux/macOS - terminal no longer stays green after osync exits
- Explicit ANSI reset sequence (
\x1b[0m) on exit prevents color leakage to shell prompt
- Shortened
- Bug Fixes
- Fixed nvidia-smi parsing for power and memory values on systems with non-English locales
- Fixed GPU stats display to properly match ollama processes to their respective GPUs by UUID
- Fixed copy command not detecting IP addresses as remote servers (e.g.,
osync cp model 192.168.0.100) - Fixed copy to remote destination requiring model name - now uses source model name when destination has no model
- Fixed HuggingFace model copy using source model name when destination has no explicit name
- Fixed
qcandbenchcommands silently exiting when remote test server is unreachable - now shows clear "Could not connect to server" error message - Fixed spurious ANSI escape characters (
←[0m) appearing after command output on Windows
- QC Command Updates
--enablethinkingand--thinklevelarguments for thinking models (qwen3, deepseek-r1)--no-unloadallargument to skip unloading all models before testing--overwriteargument to overwrite existing output file without prompting- Fixed HttpClient timeout modification error ("This instance has already started one or more requests")
- Per-request timeout handling allows dynamic timeout extension during retries
- Improved pull progress display with download speed and ETA for
--ondemandmode - Fixed OutOfMemoryException during JSON serialization of large results (uses streaming)
- Fixed model preloading hanging issue - switched from
/api/chatto lightweight/api/generatecall - Smart model loading: detects if model is already loaded, skips unload and just resets keep_alive timer
- Two-phase HuggingFace rate limit retry: 50 quick retries (2s delay), then 50 slow retries (30s/API delay)
--fixargument to recover corrupted/malformed JSON results files (outputs to .fixed.json)- Multi-strategy recovery: structural analysis finds last valid QuestionResult, then rebuilds proper JSON closures
- Handles corrupted closing sequences (e.g., missing array brackets, extra braces)
- Reports recovery statistics: truncated arrays/objects, fixed closures, removed bytes
- Auto-backup: creates .backup.zip of existing results file before continuing (protects against data loss)
- Atomic file saves: write to temp file then rename, prevents corruption on cancellation
- Force exit (double Ctrl+C) now saves results before exiting
- Fixed Spectre.Console markup errors when loading corrupted JSON files (proper escape of exception messages)
- Fixed logprobs detection after model preload by using separate HTTP connections
- QcView Command Updates
--overwriteargument to overwrite existing output file without prompting- Fixed OutOfMemoryException when loading large JSON results files (uses streaming deserialization)
- BenchView Command Updates
- Fixed OutOfMemoryException when loading large JSON results files (uses streaming deserialization)
- Multiple results files comparison - Pass comma-separated files to compare different models
- Test suite digest validation ensures all files used identical test suite
- Default output filename computed from input filenames (e.g.,
file1-file2.html) - Enhanced HTML output - qcview-style dark theme with toggle, collapsible sections
- Description field spans full width in header
- Q&A details always shown in HTML and PDF (no longer requires
--details) - Full answers, model thinking, judgment reasons, and tools used
- Subcategory table with two-row header (category spanning subcategories)
- Average speed per category table with response times
- PDF improvements - Header in table format with all metadata including versions
- All tables have proper borders, all three summary tables included
- Bench Command Updates
- Test suite digest validation - SHA256 digest computed and saved for each test suite
- Validates digest when appending to existing results file (prevents mixing test versions)
- Context length comparison fix - 2% margin tolerance for model/category matching
- Handles both 1000-based (256k=256000) and 1024-based (256k=262144) context definitions
- Models reporting 256000 now correctly match 256k category instead of falling to 128k
- Process Logging - New
--logfileargument forqcandbenchcommands- Logs all console output to a text file with timestamps
- Appends to existing log file (multiple sessions in one file)
- Strips ANSI escape codes and Spectre.Console markup for clean text output
- Timestamps each line with millisecond precision (e.g.,
[2026-01-17 06:38:20.176]) - Session start/end markers for clear log boundaries
- URL Handling Improvements - Better defaults for remote server URLs
- Default port 11434 when not specified (e.g.,
http://192.168.1.100becomeshttp://192.168.1.100:11434) - Default
http://protocol when not specified (e.g.,192.168.1.100:11434becomeshttp://192.168.1.100:11434) - IP addresses and hostnames automatically detected as remote servers across all commands
- Trailing slash indicates server URL (e.g.,
myserver/→http://myserver:11434) since model names cannot end with/ - Applied consistently to: cp, ls, rm, update, ps, load, unload, run, qc, bench
- Remote server detection rules:
- Has protocol (
http://orhttps://) → remote server - Is an IP address (
192.168.0.100) → remote server - Has port number (
host:11434) → remote server - Ends with
/(host/) → remote server - Starts with
localhost→ remote server - Plain hostname without port or
/(host) → treated as model name
- Has protocol (
- Default port 11434 when not specified (e.g.,
- Terminal Compatibility Fix - Fixed application hang with TERM=xterm-256color
- Automatic TERM override to xterm-16color on Linux/macOS in SSH/tmux sessions
- Restores original TERM on exit
- Prevents console library capability detection hangs over remote connections
- Interactive Mode Improvements - Updated REPL welcome message to mention Ctrl+C for exit
v1.2.8
- Cloud Provider Support for Judge Models - Use cloud AI providers for
--judgeand--judgebest- Support for 9 providers: Anthropic Claude, OpenAI, Google Gemini, Azure OpenAI, Mistral AI, Cohere, Together AI, HuggingFace, and Replicate
- Syntax:
@provider[:token]/model(e.g.,@claude/claude-sonnet-4-20250514,@openai/gpt-4o) - API keys loaded from environment variables by default, can be specified explicitly
- Connection and model validation before testing starts
- Cloud provider info (name, API version) recorded in results for traceability
- QcView displays cloud provider badges in HTML/PDF output (only for cloud, not Ollama)
- New
--help-cloudoption for detailed provider documentation
- PDF Text Rendering Fix - Fixed text corruption in Q&A answers for PDF output
- Resolved character scrambling issue with certain text patterns (e.g., Python format strings)
- Uses line-by-line Text elements to prevent text reordering
- Added Courier monospace font for code content for better readability
- Added text sanitization to handle problematic Unicode characters
- QcView File Access Check - Moved file overwrite confirmation before progress bar
- Prevents concurrent display errors when output file already exists
- Applies to all output formats (JSON, Markdown, HTML, PDF)
v1.2.7
- Separate Best Answer Judge Model (--judgebest) - New command-line argument for best answer determination
- Use a different model for best answer judgment vs similarity scoring (--judge)
--judgebestcan be used alone or combined with--judgefor different models- Same configuration options as
--judge: local model name orhttp://host:port/modelfor remote - New system prompt focused purely on qualitative best answer determination
- Supports both serial and parallel execution modes
- Works with
--rejudgeto re-run only best answer judgment with new model
- Version Tracking in QC Results - Record osync and Ollama versions in test results
OsyncVersion- Version of osync used for testingOllamaVersion- Ollama server version for test quantizationsOllamaJudgeVersion- Ollama version for judge server (similarity scoring)OllamaJudgeBestAnswerVersion- Ollama version for best answer judge server- Versions captured automatically from Ollama
/api/versionendpoint
- QCView Output Updates - All output formats updated with new information
- Table output shows Best Answer Judge model (when different from Judge) and versions
- JSON output includes all version fields and JudgeModelBestAnswer
- Markdown output includes Best Answer Judge and versions in header
- HTML output shows Best Answer Judge in info grid and versions row
- PDF output includes Best Answer Judge and versions in header tables
- Manage TUI Multi-Select Delete Fix - Fixed batch delete for multiple selected models
- Multi-selection delete now works correctly (previously only deleted single model)
- Added batch confirmation dialog showing count and list of models to be deleted
- Judge Retry Output Improvements - Better visibility into retry attempts during judgment
- Both judge and judgebest operations now show retry warnings with error codes at each attempt
- Displays retry delay countdown before each retry attempt
- Fixed Copy to Remote Server - Resolved HTTP 500 errors when loading copied models
- Fixed
stopparameter serialization (now correctly sent as array instead of string) - Fixed numeric/boolean parameter type conversion (top_k, temperature, seed, etc.)
- New
ConvertParameterValuehelper ensures correct JSON types for all Ollama model parameters
- Fixed
- Fixed HuggingFace Model Copy - Correct path resolution for
hf.co/...models- HuggingFace models now use correct manifest path (not under registry.ollama.ai)
- Fixed cross-platform path separator handling for model paths with forward slashes
- Load/Unload URL Format Support - Both commands now accept URL format with embedded model name
- Supports
osync load http://host:port/modelnamein addition toosync load modelname -d host - Same URL parsing for unload command
- Supports
- Fixed --rejudge Model Pulling - Rejudge mode no longer attempts to download test models
- When using
--rejudgewith existing results, only the judge model is needed - Wildcard expansion now filters to only tags present in the results file
- Skips model verification for all existing results in rejudge mode
- Properly queues partial results for re-judgment without resuming tests
- When using
v1.2.6
- QcView Multiple Output Formats - Export results to various formats
- Markdown (.md) - Tables formatted for GitHub and documentation
- HTML (.html) - Interactive report with dark/light theme toggle, collapsible Q&A sections, color-coded scores
- PDF (.pdf) - Professional report using QuestPDF library with:
- Summary tables with color-coded scores for all metrics
- Scores by category table
- Rankings tables (by score, eval speed, perplexity, best answers)
- Full Q&A pages for each quantization with judgment details
- Use
-Fo md,-Fo html, or-Fo pdfto select format
- QcView Repository URL - New
--repoargument to specify model source repository- URL is displayed in output headers and included in JSON export
- Can be saved during
qctesting and overridden inqcview
- Headless/Background Mode Fix - Fixed console errors when running qc command in background
- Console.WindowWidth and Console.ReadKey now properly handled in headless environments
- Prevents "The handle is invalid" errors when running without a terminal
- New Version Command - Added
osync version(alias-v) to display version info- Shows osync version number and build timestamp
--verboseflag displays detailed info: binary path, installation status, shell type/version, tab completion status- Detects bash, zsh, PowerShell (Core/Desktop), and cmd shells
- Smart installation detection: when running a different binary, compares version AND build timestamp with installed version
- Reports if installed version is older/newer (e.g.,
installed v1.2.6 (b20260110-1156) is older) - Fixed tab completion detection to match actual script markers in profiles
- Model Digest Tracking - QC results now include SHA256 digest for each tested model
- Full digest (
Digest) and short digest (ShortDigest, first 12 chars) stored in results JSON - Automatically populated from local Ollama or HuggingFace registry
- Backfill: missing digests are automatically retrieved when loading existing results files
- Full digest (
- Fixed Model ID Display - IDs now show first 12 chars of manifest SHA256 (matches
ollama ls)osync lsand manage TUI compute SHA256 of manifest file content- ID column width increased from 8 to 12 characters
- Consistent with
ollama lsoutput for easy cross-reference
- Improved osync ps Output - Dynamic console width and better model name display
- Detects console width and adjusts column sizes dynamically
- Model names now truncated from beginning to preserve full tag (e.g.,
...0B-A3B-Instruct-GGUF:Q4_K_S) - Better visibility of quantization tags for HuggingFace models with long paths
- Load Command Timing - Shows elapsed time and API-reported load duration
- Displays total elapsed time and Ollama's
load_durationfrom response - Example:
✓ Model 'model:tag' loaded successfully (2m 15s) (API: 2m 5s)
- Displays total elapsed time and Ollama's
- QCView Table Alignment Fix - Tag and Quant columns now left-aligned instead of centered
- Timeout Handling Improvements - Better handling of HTTP timeouts during testing
- Timeouts are now properly distinguished from user cancellation (no longer shows "Operation cancelled by user")
- Timeouts trigger retry with exponential backoff instead of immediate failure
- After retry attempts exhausted, prompts user: y=cancel, n=double timeout and retry
- Allows recovery from slow model responses without losing progress
- Improved On-Demand Model Cleanup - Fixed critical bug where models were deleted during testing
- Models with incomplete test results are NEVER cleaned up (preserves for resume)
- Fixed cleanup to protect incomplete models regardless of error type (timeout, cancellation, etc.)
- On-demand status tracking is now consistent when resuming interrupted tests
- Fixed HuggingFace Wildcard Tag Detection - Now detects all quantization formats
- Added support for XL variants (Q2_K_XL, Q3_K_XL, Q4_K_XL, Q5_K_XL, Q6_K_XL, Q8_K_XL)
- Added support for TQ ternary quantization (TQ1_0)
- Fixed HuggingFace Model Quant Column - QC results now correctly show quantization type
- Ollama returns
"quantization_level": "unknown"for HuggingFace models - Now extracts quantization type from model name/tag when API returns "unknown"
- Ollama returns
- Enhanced Quantization Display with Tensor Analysis - Quant column now shows dominant tensor quantization
- Analyzes transformer block weight tensors only (excludes embeddings, output, and norms)
- Calculates weighted percentage by tensor size (elements × bits per weight)
- Displays format like
Q4_0 (87%)orQ6_K (81% Q8_0)showing actual tensor distribution - Uses Ollama API
verbose=trueto fetch tensor metadata - Fixed: Extract quant type from model name before tensor analysis for correct formatting
- Fixed: Filter to transformer weights only (Q8_0 embeddings/output were skewing results)
- Fixed: Unknown tensor types shown with "?" suffix (e.g.,
Q3_K?) to indicate uncertainty - Supports all quantization types: Q*_K variants, IQ (importance matrix), and TQ (ternary)
- Fixed QC Model Validation - Relaxed overly strict parameter size comparison
- Parameter size formatting varies between models (e.g., "999.89M" vs "1,000M" for same model)
- Now only warns on family mismatch instead of blocking testing
- Testing continues even with warnings
- Improved Judge API Retry Strategy - More resilient handling of judge server errors
- Increased retry attempts from 5 to 25 for judge API calls
- Delay ramps from 5 seconds to 30 seconds progressively
- Shows warning and skips judgment only after all retries exhausted (instead of failing)
- Better handles overloaded or slow judge servers (HTTP 500 errors)
- Fixed Base Model Re-Pull When Adding Quants - Skip base model if results already exist
- When adding new quants to existing test results without
-b, no longer tries to pull the base model - If results file contains any base model results (even partial), the base is skipped entirely
- Improved base model detection: automatically identifies base by common patterns (fp16, f16, bf16, etc.)
- Use
--forceto re-run the base model if needed
- When adding new quants to existing test results without
- Improved osync ls Wildcard Handling - Better shell expansion handling on Linux/macOS
- Default behavior:
osync ls codematches models starting with "code" (prefix match, same ascode*) - Suffix match:
osync ls *q4_k_mfinds all models ending with "q4_k_m" (useful for finding by quantization) - Contains match:
osync ls *code*finds models containing "code" anywhere in the name - Shell expansion handling: detects when shell expanded unquoted wildcards and shows helpful warning
- Suggests using quotes to prevent expansion:
osync ls 'gemma*'
- Default behavior:
- Wildcard Tag Expansion for osync pull - Pull multiple models with tag patterns
- Supports wildcards in tags:
osync pull gemma3:1b-it-q*pulls all matching tags - Works with HuggingFace:
osync pull hf.co/unsloth/gemma-3-1b-it-GGUF:IQ2* - Works with remote servers:
osync pull -d http://server:11434 gemma3:1b-it-q* - Automatically resolves available tags from Ollama registry or HuggingFace API
- Shows list of matching tags before pulling
- Supports wildcards in tags:
- Judge Best Answer Tracking - QC judge now evaluates which response is qualitatively better
- Judge model returns
bestanswer: A (base better), B (quant better), or AB (tie) - Verbose output shows best answer for each judgment:
Score: 75% (27/50 54%) Best: AB - Handles edge cases: normalizes various formats (ResponseA, Response_A, Tie, identical, etc.)
- Results automatically re-judged if
--judgeis active andbestansweris missing
- Judge model returns
- QcView Judge Best Column - New column showing quant win statistics
- Format:
67% (B:10 A:5 =:3)showing quant won 67% of non-tie comparisons - B = quant better, A = base better, = = tie
- Best percentage excludes ties (only counts decisive wins/losses)
- Color-coded: green (>=60%), yellow (40-60%), red (<40%)
- Format:
- Enhanced JSON Output - Additional statistics in JSON export
- Per-question
BestAnswerfield (A/B/AB) - Per-quantization:
BestCount,WorstCount,TieCount,BestPercentage,WorstPercentage,TiePercentage - Per-category:
CategoryBestStatswith counts and percentages
- Per-question
- QcView Metrics-Only Mode - New
--metricsonlyargument to ignore judgment data- Shows only metrics-based scores (token similarity, logprobs divergence, perplexity, length consistency)
- Useful for comparing pure model output quality without judge influence
- Works with all output formats (table, json, md, html, pdf)
- Automatic Judge Context Length - Judge model context is now auto-calculated by default
- When
--judge-ctxsizeis 0 (new default), calculates: test_ctx × 2 + 2048 - Ensures judge has enough context for both base and quantized responses plus prompt
- Can still be manually overridden with explicit value
- When
- PDF Generation Progress Bar - Visual progress indicator when generating PDF reports
- Shows progress through Q&A pages for each quantization
- Useful for large test results files with many questions
- PDF Layout Improvements - Better page break handling in PDF reports
- Ranking tables use ShowEntire() to prevent splitting across pages
- Speed columns simplified to show only percentage (removed tok/s to prevent wrapping)
- Category scores section moves entirely to next page if it won't fit
- Rankings organized into paired rows (Final Score + Eval Speed, Perplexity + Prompt Speed, Best Answers)
- Added Prompt Speed ranking table with vs Base percentage column
- Manage TUI Batch Delete Fix - Fixed multi-selection delete not working
- Delete now properly handles multiple selected models (Ctrl+D with checkmarks)
- Confirmation dialog shows count and lists all models to be deleted
- Dialog title shows model count (e.g., "Confirm Delete (3 models)")
- Success message shows count of deleted models
- Partial success handling when some deletions fail
v1.2.5
- QC Resume Bug Fixes - Fixed critical issues with resuming from saved results files
- Fixed model name parsing when resuming: full model paths (e.g.,
hf.co/namespace/repo:tag) are now preserved correctly instead of being incorrectly derived from the-Margument - Fixed base model handling when resuming: the stored full model name is now used instead of just the tag portion
- Fixed verification loop to use stored model names from results file
- Fixed model name parsing when resuming: full model paths (e.g.,
- Cancellation Confirmation Prompt - Added y/n confirmation before cancelling QC tests
- First Ctrl+C now prompts "Cancel testing? (y/n)" instead of immediately cancelling
- Prevents accidental cancellation of long-running tests
- Second Ctrl+C still force exits immediately
v1.2.4
- New --rejudge Argument for QC Command - Re-run judgment without re-testing
- New
--rejudgeargument to re-run judgment process for existing test results - Unlike
--forcewhich re-runs both testing and judgment,--rejudgeonly re-runs judgment - Useful for re-evaluating results with a different judge model or updated prompts
- New
- Improved Judge Response Parsing - More robust handling of judge model responses
- Case-insensitive JSON property matching for Reason/reason/REASON fields
- Multiple regex patterns with increasing leniency for fallback parsing
- Truncated JSON repair to handle incomplete responses from models
- Increased num_predict from 200 to 800 to reduce truncated responses (later increased to 8192 in v1.2.9)
- Full raw JSON output displayed when reason parsing fails (for debugging)
- Improved Judge Scoring Accuracy - Fixed score interpretation issues
- Changed JSON schema score type from "integer" to "number" for better model compatibility
- Added explicit prompt instructions for 1-100 integer scoring (not 0.0-1.0 decimal)
- Score normalization to handle both 0.0-1.0 and 1-100 ranges from different models
- Fixed HuggingFace Model Verification in On-Demand Mode - Registry check now supports HuggingFace models
- On-demand mode (
--ondemand) now properly verifies HuggingFace models (hf.co/...) - Checks HuggingFace API to verify repository and GGUF file existence
- Supports various GGUF filename patterns for tag matching
- On-demand mode (
- Fixed Base Model Name Handling - Full model names now preserved for base tag
- Base model specified as full model name (e.g.,
-b qwen3-coder:30b-a3b-fp16or-b hf.co/namespace/repo:tag) is now used as-is - Previously, only the tag portion was extracted and combined with
-Mmodel name
- Base model specified as full model name (e.g.,
- Wildcard Tag Selection for QC Command - Dynamically select quantizations with patterns
- Support for wildcard patterns (
*) in-Qargument (e.g.,Q4*,IQ*,*) - Fetches available tags from HuggingFace API for
hf.co/...models - Scrapes available tags from Ollama website for Ollama registry models
- Case-insensitive pattern matching
- New
ModelTagResolverclass for reusable tag resolution across commands
- Support for wildcard patterns (
- Improved On-Demand Cleanup - Models pulled on-demand are now properly cleaned up on failure
- On-demand models are removed when testing fails or is cancelled
- Cleanup happens in exception handlers to ensure no orphaned models remain
- Models tracked at class level for reliable cleanup across failures
- Preload failures now also trigger cleanup of on-demand models
- Improved Model Preload - Better error handling and retry logic for model loading
- Added retry logic (3 attempts with exponential backoff) for transient failures
- Shows actual error message when preload fails (HTTP status, error details)
- Uses configurable timeout (
--timeout) for model loading - Handles timeout, connection errors, and server errors gracefully
- Fixed Model Name Case Sensitivity - Handle Ollama storing HuggingFace tags with different case
- After pulling, resolves actual model name stored by Ollama (case-insensitive lookup)
- Fixes issue where
Q4_0is stored asq4_0causing preload to fail
v1.2.3
- On-Demand Mode for QC Command - Pull models automatically and remove after testing
- New
--ondemandargument to enable on-demand model management - Models missing from the Ollama instance are automatically pulled from the registry
- Models that were already present are NOT removed (only on-demand pulled models)
- After testing and judgment complete, on-demand models are removed to free disk space
- State is persisted in results file for proper cleanup on resume
- Works with both local and remote Ollama servers
- Ideal for testing large models or many quantizations without consuming permanent storage
- New
- Context Length Support for QC Command - Configure context length (num_ctx) for testing and judgment
- Default test context length: 4096, default judge context length: 12288
- Suite-level
contextLengthproperty in built-in test suites (v1base, v1quick, v1code) - External JSON test suites support
contextLengthat suite, category, and question levels - Hierarchical override system: question > category > suite
- Console output displays context length at startup and when overridden
- New
--judge-ctxsizeargument to configure judge model context length
- Improved Console Output - Context length settings displayed during test execution
- Shows test and judge context lengths at the beginning of testing
- Displays override notifications when context length changes (e.g., "Context length changed to 8192 (from category Code)")
- Fixed Linux Terminal Display Issues - Resolved ANSI color rendering problems
- Fixed white box display issue in interactive REPL mode on Linux terminals
- Downgraded PrettyConsole (3.1.0 → 2.0.0) and Spectre.Console (0.54.0 → 0.49.1) for compatibility
- Improved Installer - Streamlined installation process
- Installer now only copies the main executable (no longer copies all directory files)
- Added mechanism for platform-specific optional dependencies
- Removed unnecessary libuv.dylib dependency for macOS (not needed in .NET 8+)
- Fixed Bash Completion on Linux
- Fixed tab completion for model names containing colons (e.g.,
osync ls qwen2:) - Fixed file tab completion for qcview command
- Fixed tab completion for model names containing colons (e.g.,
v1.2.2
- Improved Judgment Prompt Format - Better compatibility with more models
- Instructions now in both system prompt and user message for redundancy
- Clear text markers for RESPONSE A and RESPONSE B instead of JSON encoding
- Question included for context with clear delimiters
- Explicit rules to prevent models from judging quality/correctness instead of similarity
- Verbose Judgment Output - New
--verboseflag to show judgment details- Displays question ID, score (color-coded), and first 4 lines of reason
- Works with both serial and parallel judgment modes
- Helps debug and understand judge model scoring
- Fixed Parallel Verbose Output - Verbose output now displays during parallel judgment execution
- Previously showed all results after completion; now shows each result as it completes
- Fixed Serial Verbose Progress - Progress bar now displays alongside verbose output in serial mode
- Improved Cancellation Handling - Ctrl+C now immediately stops judgment without retrying
- Cancellation exceptions are no longer retried 5 times
- Judgment loop checks for cancellation before each question
- Missing Reason Retry - Judge API retries up to 5 times when response contains score but no reason
- Warning displayed if reason still missing after all retries
v1.2.1
- Bug Fix: Base Model Detection - Fixed issue where base model wasn't correctly identified when using full model names
- Base tag is now properly normalized when specified with full path (e.g.,
user/model:tag) - Existing results files with missing
IsBaseflag are automatically repaired on load - Judgment now correctly runs for quantizations that need it
- Base tag is now properly normalized when specified with full path (e.g.,
- Bug Fix: Output Filename Sanitization - Model names with
/or\are now converted to-in default output filename- Prevents file path issues when model name contains directory separators
- Improved Startup Output - Output file path is now displayed early in the execution
- Shows right after loading test suite, before judge model verification
- Cancellation Improvements - Better handling of Ctrl+C during API calls
- Cancellation token now passed to HTTP requests for immediate cancellation
- Wrapped cancellation exceptions are properly detected
v1.2.0
- Coding Test Suite - New
v1codetest suite for evaluating code generation quality- 50 challenging coding questions across 5 languages: Python, C++, C#, TypeScript, Rust
- Double token output limit (8192) for longer code responses
- Questions include instruction to limit response size
- Available as
-T v1codeor via externalv1code.jsonfile
- Configurable Token Output - Test suites now support custom
numPredictvalues- Each test suite can specify its own maximum token output
- External JSON test suites support
numPredictproperty (default: 4096) - Displayed in test suite info when non-default value is used
- Improved Model Existence Check - Pull command now uses Ollama registry API for faster, more reliable model validation
- Uses
registry.ollama.ai/v2/manifest endpoint instead of HTML scraping - Properly handles both library models and user models
- Faster response times and more accurate error messages
- Uses
- True Independent Parallel Judgment - Testing continues to next quantization while judgment runs in background
- Testing no longer waits for judgment to complete before moving to next quantization
- Background judgment tasks tracked and awaited at the end with progress display
- Progress bars show real-time status for both testing and judgment
- Improved Progress Display - Better visibility into parallel operations
- Dual progress bars during testing (Testing + Judging) in parallel mode
- Background judgment status shown after each quantization completes
- Final wait screen shows progress for all pending judgment tasks
- Configurable API Timeout - Added
--timeoutargument for testing and judgment API calls- Default increased from 300 to 600 seconds for longer code generation
- Configurable via
--timeout <seconds>argument - Applies to both test model and judge model API calls
- Resume Support - Gracefully handle interruptions and resume from where you left off
- Press Ctrl+C to save partial results and exit cleanly
- Re-run the same command to resume testing from the last saved question
- Partial quantization results are preserved in the JSON file
- Progress bar shows resumed position when continuing
- Missing judgments are automatically detected and re-run on resume
- UI Improvements
- Unified color scheme: lime for good scores (80%+) and performance above 100%
- Orange color for performance below 100%
v1.1.9
- Judge Model Scoring - Use a second LLM to evaluate quantization quality
--judge <model>- Specify local or remote judge model--mode serial|parallel- Serial (after each quant) or parallel (concurrent) execution- Judge evaluates similarity between base and quantized responses (score 1-100)
- Final score: 50% metrics + 50% judgment when enabled
- Supports local and remote judge models with auto-completion
- Results display shows separate Final, Metrics, and Judge scores
- True Parallel Judgment Mode - Testing and judgment now run concurrently at the question level
- In
--mode parallel, each question is immediately judged in the background as soon as testing completes - Test model continues generating answers while judge evaluates previous questions
- Significantly reduces total execution time when using a remote judge or fast judge model
- Parallel judgment also applies to existing quantizations that need re-judging
- In
- Improved Judge Response Handling
- Changed to
/api/chatendpoint with structured JSON output schema - Added
Reasonfield to capture judge's reasoning for each score - Robust parsing with fallbacks for different model response formats
- Treats score 0 as score 1 (minimum) for models that ignore instructions
- Changed to
- Enhanced Retry Logic
- Fixed retry display to show "Attempt 2/5", "Attempt 3/5" etc. after failures
- Added detailed error messages for judge API failures
- 5 retry attempts with exponential backoff
- Improved qcview Output - Enhanced file export functionality
- Fixed table file output with correct eval/prompt speed formatting
- Fixed JSON console output avoiding Spectre.Console markup parsing errors
- Added file size display on successful file creation
- Positional argument for results file (e.g.,
osync qcview results.json) - Displays judge model info when judgment scoring is available
- Bug Fixes
- Fixed judgment skip logic: existing quantizations without judgments or with different judge model are now properly re-judged
- Fixed parallel mode not executing judgment concurrently with testing
- Documentation Updates
- Updated color coding documentation to match actual thresholds
- Clarified qcview command usage and options
v1.1.8
- External Test Suite Support - Load custom test suites from JSON files using
-T path/to/suite.json- Bundled
v1base.jsonfile for reference and portability - Create custom test suites with your own categories and questions
- Full JSON schema support with categories, questions, and metadata
- Bundled
- Force Re-run Option - Added
--forceflag to re-run testing for quantizations already in results file - Improved Base Model Handling - Base tag defaults to
fp16and is automatically detected from existing results - Bug Fixes
- Fixed case-sensitive model name matching when retrieving disk size from
/api/tags - Fixed command argument positioning (PowerArgs uses 1-based indexing)
- Fixed
osync lsto accept model name as positional argument (e.g.,osync ls qwworks likeosync ls qw*)
- Fixed case-sensitive model name matching when retrieving disk size from
v1.1.7
- Quantization Comparison Testing - New
qcandqcviewcommands for comprehensive quality testing- 50-question test suite across 5 categories (Reasoning, Math, Finance, Technology, Science)
- 4-component weighted scoring: Logprobs Divergence (70%), Perplexity (20%), Token Similarity (5%), Length Consistency (5%)
- Performance metrics: eval/prompt tokens per second with percentage comparison to base model
- Incremental testing support - add new quantizations without re-testing existing ones
- JSON results file format with full question/answer history
- Color-coded table output in
qcview(90-100% green, 80-90% lime, 70-80% yellow, 50-70% orange, <50% red) - JSON export option for programmatic analysis
- Uses Ollama's logprobs API (requires Ollama v0.12.11+)
- Configurable test parameters: temperature, seed, top_p, top_k, repeat_penalty, frequency_penalty
- Works on both local and remote servers
- Bug Fixes
- Fixed PowerArgs duplicate alias errors by removing explicit shortcuts (auto-generated from property names)
- Fixed model detection issue:
/api/showendpoint doesn't includesizefield, now fetches from/api/tags - Fixed misleading "Results saved" message when no quantizations were successfully tested
- Added proper error handling with exit codes (returns 1 when no results)
- Set max output tokens to 4096 to prevent tests hanging on overly verbose answers
- Improved tag handling: accepts any tag format (with or without ":"), extracts quantization from API
v1.1.6
- Manage TUI Command - Full-screen interactive terminal user interface for model management
- Real-time model listing with dynamic column widths
- Multi-selection support for batch operations (copy, update, delete)
- Live filtering with instant search
- Six sort modes: Name+/-, Size+/-, Created+/- (with visual status indicator)
- Seven built-in themes (Default, Dark, Blue, Solarized, Gruvbox, Nord, Dracula)
- All operations accessible via keyboard shortcuts (Ctrl+C/M/R/S/D/U/P/L/K/X/O/T/Q)
- Enter key support in all dialogs for faster workflow
- Smart model selection persistence after operations
- Batch copy with incremental uploads
- Console output for long operations (update, pull) with auto-return to TUI
- Process Status Enhancements -
pscommand improvements- Shows VRAM usage percentage when model is partially loaded (e.g., "1.33 GB (63%)")
- Consistent tabular format in both CLI and manage TUI
- Displays model ID, size, context length, and expiration time
- Load/Unload Commands - VRAM memory management
loadcommand to preload models into memory with configurable keep-aliveunloadcommand to free VRAM immediately- Works on both local and remote servers
- Run/Chat Command - Interactive conversation with models
- Preloads model into memory before starting chat
- Shows loading status and model information
- Type
/byeor Ctrl+D to exit - Works on both local and remote servers
- Show Command - Display detailed model information
- Shows model metadata, configuration, and parameters
- Works on both local and remote servers
- Pull Command - Download models from Ollama registry
- Validates model existence on ollama.com before pulling
- Real-time progress display
- Supports library and user models
- Works on both local and remote servers
v1.1.0
- Remote-to-Remote Copy - Transfer models directly between remote servers without local storage
- Memory-Buffered Streaming - Efficient transfer using configurable memory buffer (default: 512MB)
- Simultaneous Download/Upload - Downloads from registry while uploading to destination
- Backpressure Control - Automatically throttles download when upload is slower
- Configurable Buffer Size - Use
-BufferSizeparameter (e.g.,256MB,1GB) - Enhanced Progress Display - Tqdm progress bar for remote-to-remote transfers matching local copy format
- Registry-Based Transfer - Pulls model blobs from Ollama registry for remote-to-remote operations
- Fixed HttpClient Issues - Resolved BaseAddress errors in list and copy commands
- Improved Error Messages - Clear documentation of remote-to-remote limitations
v1.0.9
- Added update command to update models to their latest versions
- Supports updating all models or specific patterns with wildcard matching
- Works on both local and remote Ollama servers
- Shows real-time progress and indicates whether each model was updated or already up to date
- Default behavior updates all models when no pattern specified
v1.0.8
- Added rename command with aliases
renandmvfor safe model renaming - Added local copy support - copy models locally, not just to remote servers
- Added sorting options for list command:
--size,--sizeasc,--time,--timeasc - Added destination existence checks for copy and rename operations to prevent overwriting
- Comprehensive README documentation with usage examples and patterns
v1.0.7
- Added list command with pattern matching support using
*wildcard - Remote listing support - list models on remote Ollama servers
- Multi-registry support - properly scans registry.ollama.ai, hf.co, hub, and custom registries
- Automatic
:latesttag fallback for copy and remove commands when tag not specified - Enhanced remove command with pattern matching support
- Fixed Ollama API migration for model creation (modelfile → files format)
- Fixed missing models in list output (models with
/in names)
v1.0.6
- Changed syntax to support multiple actions: now copying a model needs
copy(aliascp) - Fixed upload progress bar updates
- Handles automatically
latesttag if none specified
v1.0.5
- Added local models TabCompletion with interactive prompt if called without arguments
- Fixed 100 seconds timeout
- Added arguments exception handling
v1.0.4
- Check remote ollama version and display its version
- Fixed streaming output from create model
v1.0.3
- Added -bt switch to throttle the bandwidth in B, KB, MB, GB per second, eg. for 75MB/s use
-bt 75MB
v1.0.2
- Fixed build with single portable file for Linux/MacOS
v1.0.1
- Fixed bug with stdErr redirect
- Remove Linux and MacOS colored output
v1.0.0
- Initial release
Roadmap
See the open issues for a list of proposed features (and known issues).
- Top Feature Requests (Add your votes using the 👍 reaction)
- Top Bugs (Add your votes using the 👍 reaction)
- Newest Bugs
Support
License
This project is licensed under the MIT license.
See LICENSE for more information.
Similar repos
Languages
C#
98.1%
Gherkin
1.6%
Transfer a local ollama model to a remote ollama server
C#
15
54 commits
updated Oct 3, 2026
README
About
osync is a powerful command-line tool for managing Ollama models across local and remote servers.
Key Features
- 🖥️ Interactive TUI - Full-screen terminal interface with keyboard shortcuts for all operations
- 🚀 Fast Model Transfer - Copy models between local and remote Ollama servers with high-speed uploads
- 🔄 Remote-to-Remote Copy - Transfer models directly between remote servers with memory-buffered streaming
- 📋 Smart Model Management - List, copy, rename, and delete models with pattern matching support
- 🔄 Incremental Uploads - Skips already transferred layers, saving bandwidth and time
- 📊 Progress Tracking - Real-time progress bars with transfer speed indicators
- 🏷️ Auto-tagging - Automatically applies
:latesttag when not specified - 🔍 Advanced Listing - Sort models by name, size, or modification time with visual indicators
- 🌐 Multi-registry Support - Works with registry.ollama.ai, hf.co, and custom registries
- 💾 Offline Deployment - Perfect for air-gapped servers and isolated networks
- 🎯 Wildcard Patterns - Use
*wildcards for batch operations - ⚡ Bandwidth Control - Throttle upload speeds and configure memory buffer size
- 🎨 Theme Support - 34 true-color themes (Dracula, Nord, Tokyo Night, Catppuccin, Gruvbox, Rose Pine, Kanagawa, Solarized, light themes, ...) for
manageand the colored command output, adapted to 256 and 16-color terminals - 💬 Interactive Chat - Chat with models directly from the CLI
- 🧠 Memory Management - Load/unload models from VRAM with process status monitoring
- 📊 Quantization Comparison - Compare quality and performance across model quantizations with detailed scoring
- 🔬 Context Benchmark - Test model context tracking with dynamic story-based benchmarks, tools, custom test suites
- 📈 Real-Time Monitor - Live dashboard with GPU/CPU metrics, VRAM usage graphs, model status, and Ollama process tracking
- 🔀 Ollama and xOllama - Works with Ollama and the xOllama fork (see Local server)
Built With
[C# .NET 10]
Getting Started
Prerequisites
[Windows/Linux/MacOS]
[Arm64/x64/Mac]
Installation
[Download latest binary release]
[Build from sources]
Clone the repo
dotnet build(any OS, .NET 10 SDK) or Visual Studio 2022;dotnet publish osync/osync.csproj -c Release -r <win-x64|linux-x64|osx-arm64|osx-x64>for a single-file executable. See docs/DEVELOPMENT.md for tests, CI and releases.
Local server (Ollama and xOllama)
Commands without -d work on the local server, found in this order:
XOLLAMA_HOST(xOllama), thenOLLAMA_HOST(Ollama); bind addresses such as0.0.0.0are mapped tolocalhost. With"ignoreEnvironment": true(osync setup server env ignore, or the questionosync setup serverasks when one of them is set) the configured server comes first- The server in the preferences file (
osync setup server) - Otherwise the first server answering on
localhost:11434(Ollama default) orlocalhost:22434(xOllama default)
osync install configures it: when exactly one server answers on this machine it is used without questions; when none or both answer, it asks for the server type (1 Ollama, 2 xOllama, 3 both side by side), host and ports. With both, one is the default local server and each gets an alias (ollama, xollama), so the other one is always at hand: osync ls -d xollama, osync cp model xollama/. Change it any time with osync setup server.
osync detects whether a server is Ollama or xOllama (osync ps shows it) and, for local operations, runs the matching CLI: xollama when the local server is xOllama (or only xollama is installed), else ollama. Set OSYNC_OLLAMA_CLI to force a specific CLI. The models directory is taken from XOLLAMA_MODELS, then OLLAMA_MODELS, then the platform default.
Remote servers given without a port use :11434, or :22434 when the host only answers there (xOllama).
Preferences file
osync keeps user preferences in settings.json in the per-OS configuration folder (osync -v --verbose shows the path):
| OS | Location |
|---|---|
| Windows | %APPDATA%\osync\settings.json |
| macOS | ~/Library/Application Support/osync/settings.json |
| Linux | $XDG_CONFIG_HOME/osync/settings.json (default ~/.config/osync/settings.json) |
{
"server": { "flavor": "ollama", "host": "localhost", "both": true },
"aliases": {
"ollama": "http://localhost:11434",
"xollama": "http://localhost:22434",
"gpu": "http://192.168.1.10:11434"
},
"colorMode": "auto",
"manage": { "theme": "Dracula", "sort": "size-", "servers": [ "gpu" ] },
"shell": { "theme": "Tokyo Night" }
}
server- the local server:flavor(auto,ollama,xollama),hostandport(default: the flavor's port);bothwhen Ollama and xOllama run side by side (flavoris then the default one).XOLLAMA_HOST/OLLAMA_HOSToverride it, unlessignoreEnvironmentistrue.aliases- server aliases.colorMode-auto(detect),truecolor,256,16ornone;OSYNC_COLOR_MODEandNO_COLORoverride it.manage.theme,manage.sort- theme (also chosen with Ctrl+T inmanage) and initial sort order (name+,name-,size-,size+,created-,created+) ofosync manage.manage.servers- aliasesmanageswitches to with Ctrl+Left / Ctrl+Right, after the local server (with Ollama and xOllama side by side the other one is always included).shell.theme- colors of the command output (any theme name, orplainfor no colors).OSYNC_CONFIG_DIRmoves the settings folder.
Everything can be changed with osync setup instead of editing the file.
Server aliases
An alias is a short name for a server (osync setup alias add gpu 192.168.1.10), usable wherever osync expects a server:
osync ls -d gpu # list the models of the server
osync cp qwen3:8b gpu/ # upload
osync cp gpu/qwen3:8b qwen3-gpu # download
osync manage gpu
An alias takes precedence over a model namespace with the same name (gpu/model). Names start with a letter and use letters, digits, - and _.
Colors
osync detects the terminal's color depth from COLORTERM (truecolor/24bit), TERM (*-256color, *-direct), TERM_PROGRAM and Windows Terminal, and uses true color, 256 or 16 colors accordingly (osync -v --verbose shows what was detected and why). SSH does not forward COLORTERM by default, so terminals that support true color are seen as 256-color over SSH/tmux: set "colorMode": "truecolor" in the settings file, or OSYNC_COLOR_MODE=truecolor, or forward COLORTERM (SendEnv COLORTERM / AcceptEnv COLORTERM). NO_COLOR disables colors.
Command output is colored with the shell theme (osync setup shell theme NAME; osync setup shell themes shows every theme with a preview): the same 34 themes as manage, with light themes for light terminals (the default follows COLORFGBG when the terminal sets it), or plain. Output that goes to a pipe or a file is never colored.
osync manage draws with 24-bit colors; at 256 colors every theme color is snapped to the xterm-256 palette (tmux and most 256-color terminals then show exactly that color), at 16 colors the themes switch to the 16 standard colors with contrast checks. macOS Terminal.app gets 16 colors unless colorMode says truecolor (it misreads 24-bit colors on older macOS). If manage does not start or draw correctly on an old Windows console, OSYNC_TUI_DRIVER=windows selects the Windows console driver (ansi and dotnet are the others).
Usage
DeepWiki
An AI generated DeepWiki is available here:
https://deepwiki.com/mann1x/osync
Quick Start
# Interactive TUI for model management (recommended)
osync manage
# Copy local model to remote server
osync cp llama3 http://192.168.100.100:11434
# Copy to remote server (short forms - port 11434 and http:// are defaults)
osync cp llama3 192.168.100.100 # IP address detected as server
osync cp llama3 myserver:11434 # hostname with port
osync cp llama3 myserver/ # trailing slash indicates server
# List all local models
osync ls
# Chat with a model
osync run llama3
# Interactive REPL mode with tab completion
osync
Commands
Copy (cp)
Copy models locally, to remote servers, or between remote servers.
# Local copy (create backup)
osync cp llama3 my-backup-llama3
osync cp llama3:70b llama3:backup-v1
# Local to remote (upload to server) - multiple ways to specify server
osync cp llama3 http://192.168.0.100:11434 # Full URL
osync cp llama3 192.168.0.100 # IP address (auto: http:// + :11434)
osync cp llama3 192.168.0.100:11434 # IP with port (auto: http://)
osync cp llama3 myserver:11434 # Hostname with port
osync cp llama3 myserver/ # Trailing slash = server
# Copy HuggingFace model to remote (uses source model name)
osync cp hf.co/unsloth/gemma-3-1b-it-GGUF:Q4_K_M 192.168.0.100
# Remote to remote (transfer between servers)
osync cp http://192.168.0.100:11434/qwen2:7b http://192.168.0.200:11434/qwen2:latest
osync cp http://server1:11434/llama3 http://server2:11434/llama3-copy
# Remote to local (download from a server to the local server)
osync cp http://192.168.0.100:11434/my-finetune my-finetune
# With custom memory buffer size (default: 512MB)
osync cp http://server1:11434/llama3 http://server2:11434/llama3 -BufferSize 256MB
osync cp http://server1:11434/qwen2 http://server2:11434/qwen2 -BufferSize 1GB
# With bandwidth throttling
osync cp llama3 http://192.168.0.100:11434 -bt 50MB
Features:
- Automatic
:latesttag when not specified - Prevents overwriting existing models
- Skips already uploaded layers
- Real-time progress with transfer speed
- Memory-buffered streaming for remote-to-remote transfers
- Simultaneous download and upload with backpressure control
- Smart server detection: IP addresses,
hostname:port, orhostname/auto-detected as remote servers - Uses source model name when destination server has no model specified
Remote-to-remote and remote-to-local copies (push relay):
Ollama has no API to download a model, but a server can push a model to a registry. For copies out of a server, osync runs a temporary registry endpoint (the relay) on the machine where osync runs:
- the source server pushes the model to the relay under a temporary name;
- the relay streams every blob straight into the destination server (no disk use, memory bounded by
-BufferSize,-btthrottling applies); blobs the destination already has are skipped; - the destination installs the model's manifest from the relay, so the copy is complete: weights, projector, template, parameters, system prompt, license, messages;
- the temporary names are deleted on both servers.
This copies any model the source has, including models you created or imported, works between Ollama and xOllama in both directions, and needs no internet access.
Requirements and options:
- The source server must be able to connect to the machine running osync (and the destination should too, to install the manifest; otherwise osync recreates the model from its files). The relay listens on a free port chosen by the OS (above 1024), or on
OSYNC_RELAY_PORT; open your firewall accordingly. A Windows destination cannot store a model named after the relay'shost:port, so the model is recreated there from its manifest (byte for byte); a Windows source needs the relay on port 80, which osync then uses for that copy if it is free. OSYNC_RELAY_HOST- address the servers should use to reach osync (default: the local address that routes to the source server), e.g. behind NATOSYNC_RELAY_PORT- fixed relay port (e.g. one opened in the firewall)OSYNC_RELAY_INSTALL=create- skip installing the manifest from the relay and recreate the model with/api/create(for destinations that cannot connect to osync)- If the source server cannot reach the relay, osync falls back to downloading the blobs from registry.ollama.ai, which only works for models pulled from the Ollama registry.
List (ls)
List models with filtering and sorting options.
# List all local models
osync ls
# List models matching pattern
osync ls "llama*"
osync ls "*:7b"
osync ls "mannix/*"
# List remote models
osync ls http://192.168.0.100:11434 # Full URL
osync ls 192.168.0.100 # IP address (auto: http:// + :11434)
osync ls myserver/ # Hostname with trailing slash
osync ls "qwen*" -d 192.168.0.100 # Filter with pattern on remote
# Sort by size (descending)
osync ls --size
# Sort by size (ascending)
osync ls --sizeasc
# Sort by modified time (newest first)
osync ls --time
# Sort by modified time (oldest first)
osync ls --timeasc
Output:
NAME ID SIZE MODIFIED
llama3:latest 365c0bd3c000 5 GB 2 months ago
qwen2:7b 648f809ced2b 4 GB 1 years ago
mistral:latest 2ae6f6dd7a3d 4 GB 1 years ago
Rename (rename, mv, ren)
Rename models safely by copying and deleting the original.
# Rename with implicit :latest tag
osync rename llama3 my-llama3
osync mv llama3 my-llama3
# Rename with explicit tags
osync ren llama3:7b my-custom-llama:v1
# Create versioned backup
osync mv qwen2 qwen2:backup-20241218
Features:
- Three-step process: copy → verify → delete
- Checks if destination already exists
- Only deletes original after successful verification
Remove (rm, delete, del)
Delete models with pattern matching.
# Delete specific model
osync rm tinyllama
osync rm llama3:7b
# Delete with pattern
osync rm "test-*"
osync rm "*:backup"
# Delete from remote server
osync rm "old-model*" http://192.168.0.100:11434
Features:
- Automatic
:latesttag fallback - Wildcard pattern support
- Confirmation before deletion
- Works on local and remote servers
Update (update)
Update models to their latest versions locally or on remote servers.
# Update all local models
osync update
osync update "*"
# Update specific model
osync update llama3
osync update llama3:latest
# Update models matching pattern
osync update "llama*"
osync update "*:7b"
osync update "hf.co/unsloth/*"
# Update all models on remote server
osync update http://192.168.0.100:11434
osync update "*" http://192.168.0.100:11434
# Update specific models on remote server
osync update "llama*" http://192.168.0.100:11434
Features:
- Updates models to latest available versions
- Shows real-time progress during updates
- Indicates whether each model was updated or already up to date
- Supports wildcard pattern matching
- Works on both local and remote servers
- Default pattern is
*(all models) when not specified
Output:
Updating 2 model(s)...
Updating 'llama3:latest'...
pulling manifest
pulling 6a0746a1ec1a... 100% ▕████████████████▏ 4.7 GB
✓ 'llama3:latest' updated successfully
Updating 'qwen2:7b'...
✓ 'qwen2:7b' is already up to date
Show (show)
Display detailed information about a model.
# Show information about a local model
osync show llama3
osync show qwen2:7b
# Show information about a remote model
osync show llama3 -d http://192.168.0.100:11434
# Show specific information
osync show llama3 --license # Show license only
osync show llama3 --modelfile # Show modelfile only
osync show llama3 --parameters # Show parameters only
osync show llama3 --system # Show system prompt only
osync show llama3 --template # Show template only
osync show llama3 -v # Show all information
Options:
<model>- Model name to show information (required)-d <url>- Remote server URL (default: local)--license- Show license information--modelfile- Show modelfile--parameters- Show parameters--system- Show system prompt--template- Show template-v, --verbose- Show all information
Features:
- Displays model metadata and configuration
- Shows model file details, parameters, and system information
- Works on both local and remote servers
Pull (pull)
Pull a model from the Ollama registry.
# Pull a model from the registry
osync pull llama3
osync pull qwen2:7b
osync pull hf.co/unsloth/llama3
# Pull to a remote server
osync pull llama3 http://192.168.0.100:11434
Features:
- Downloads models from Ollama registry
- Shows real-time progress during download
- Supports library models and user models
- Works on both local and remote servers
Run (run, chat)
Interactive chat with a model.
# Chat with a local model
osync run llama3
osync chat qwen2:7b
# Chat with a model on remote server
osync run llama3 -d http://192.168.0.100:11434
# With extended thinking mode (for reasoning models)
osync run qwen3 --think medium
# With verbose output (shows timing stats)
osync run llama3 --verbose
# Vision models with custom image dimensions
osync run llava --dimensions 512
Options:
<model>- Model name to run/chat with (required)-d <url>- Remote server URL (default: local)--format <value>- Response format (e.g., json)--keepalive <duration>- Keep alive duration (e.g., 5m, 1h, default: server default)--nowordwrap- Disable word wrap--verbose- Show verbose output (timing stats)--dimensions <value>- Image dimensions for vision models (e.g., 512)--hidethinking- Hide thinking process output--insecure- Allow insecure connections--think <level>- Enable extended thinking (reasoning) mode with level (low, medium, high)--truncate- Truncate long context (default: server setting)
Features:
- Interactive conversation mode
- Preloads model into memory before chat
- Shows model loading status
- Extended thinking support for reasoning models (qwen3, deepseek-r1, etc.)
- Type
/byeor press Ctrl+D to exit - Works on both local and remote servers
Process Status (ps)
Show models currently loaded in memory, plus system monitoring when running locally.
# Show loaded models on local server (includes GPU and process stats)
osync ps
# Show loaded models on remote server (no GPU/process stats)
osync ps http://192.168.0.100:11434 # Full URL
osync ps 192.168.0.100 # IP address (auto: http:// + :11434)
osync ps myserver/ # Hostname with trailing slash
osync ps -d myserver:11434 # Using -d flag with port
Features:
- Displays models loaded in VRAM
- Shows VRAM usage with percentage when partially loaded
- Displays model size, context length, and expiration time
- Formatted table output
- Local only: Ollama process CPU and memory usage
- Local only: GPU monitoring (NVIDIA via nvidia-smi, AMD via rocm-smi)
Output (local server):
Loaded Models:
---------------------------------------------------------------------------------------------------------------------------------------
NAME ID SIZE VRAM USAGE CONTEXT UNTIL
---------------------------------------------------------------------------------------------------------------------------------------
tinyllama:1.1b-chat-v1-fp16 71c2f9b69b52 2.11 GB (1B) 1.33 GB (63%) 4096 4 minutes from now
---------------------------------------------------------------------------------------------------------------------------------------
Ollama Process:
PID: 12345 CPU: 2.3% Memory: 1.2 GB (Working Set)
GPU Status (NVIDIA):
GPU 0: NVIDIA GeForce RTX 4090
Utilization: 45% Memory: 8192 MB / 24576 MB (33%) Temp: 65°C Power: 250 W / 450 W
GPU Monitoring:
- Automatically detects nvidia-smi (NVIDIA) or rocm-smi (AMD)
- Shows GPU utilization, memory usage, temperature, and power draw
- Supports multiple GPUs
Real-Time Monitor (psmonitor, monitor, psm)
Interactive real-time monitoring dashboard inspired by nvitop, with braille-dot graphs and color-coded metrics.
# Start monitor with default settings (5s refresh, 5m history)
osync psmonitor
osync monitor
osync psm
# Custom refresh interval (2 seconds)
osync psmonitor 2s
osync psmonitor -L 2
# Custom history duration (10 minutes) - multiple formats supported
osync psmonitor -Hi 10 # Plain integer = minutes (10 minutes)
osync psmonitor -Hi 10m # Explicit minutes
osync psmonitor --history 30m # 30 minutes
osync psmonitor -Hi 1h # 1 hour
osync psmonitor -Hi 1h30m # 1 hour 30 minutes
# Combined options
osync psmonitor -L 2 -Hi 15 # 2s refresh, 15 minutes history
# Monitor remote server (shows loaded models only, no GPU/system stats)
osync psmonitor -d http://192.168.0.100:11434
Options:
-d <url>- Remote server URL (default: local). Remote mode shows loaded models only, no GPU/system stats-L <interval>- Refresh interval (default: 5s). Supports: 5, 5s, 30s, 1m, 5m, 1h30m-Hi, --history <duration>- Initial graph history duration (default: 5m). Plain integers are treated as minutes (e.g., -Hi 10 = 10 minutes). Supports: 1m, 30m, 1h, 1h30m
Features:
- Real-time GPU utilization graphs with braille characters
- CPU usage history graph
- Color-coded metrics (green < 50%, yellow 50-80%, red > 80%)
- GPU metrics: utilization, VRAM, temperature, power, clocks
- System metrics: CPU, RAM, Swap with progress bars
- Loaded Ollama models with expiration times
- NVIDIA driver and CUDA version display
Interactive Controls:
- Q / Esc - Quit monitor
- P - Pause/Resume updates
- H - Show help
- ←/→ - Adjust graph history (+/- 1 minute, range: 1-60m)
- ↑/↓ - Adjust refresh interval (+/- 1 second)
- Ctrl+C - Exit
Windows Terminal Font Requirements:
The monitor uses Unicode braille characters (U+2800-U+28FF) for graphs. On Windows, you need a font that supports these characters:
-
Recommended fonts:
- Cascadia Code / Cascadia Mono (included with Windows Terminal)
- DejaVu Sans Mono
- Consolas (partial support)
- NSimSun
-
To change font in Windows Terminal:
- Open Windows Terminal settings (Ctrl+,)
- Select your profile (e.g., "Command Prompt" or "PowerShell")
- Go to "Appearance" → "Font face"
- Select "Cascadia Code" or "Cascadia Mono"
-
Note: CMD with default raster fonts will show "?" for braille characters. Use Windows Terminal for best results.
Load (load)
Preload a model into memory.
# Load a model on local server
osync load llama3
osync load qwen2:7b
# Load a model on remote server
osync load llama3 -d http://192.168.0.100:11434
# URL format with embedded model name
osync load http://192.168.0.100:11434/llama3
Options:
<model>- Model name to load into memory (required)-d <url>- Remote server URL (default: local)
Features:
- Preloads model into VRAM for faster inference
- Shows elapsed time and API-reported load duration
- Works on both local and remote servers
Unload (unload)
Unload a model from memory.
# Unload a specific model on local server
osync unload llama3
osync unload qwen2:7b
# Unload all models (no model name specified)
osync unload
# Unload a model on remote server
osync unload llama3 -d http://192.168.0.100:11434
# URL format with embedded model name
osync unload http://192.168.0.100:11434/llama3
Options:
<model>- Model name to unload from memory (optional - if not specified, unloads all models)-d <url>- Remote server URL (default: local)
Features:
- Frees VRAM by unloading model
- Immediate unloading (keep-alive set to 0)
- Works on both local and remote servers
Quantization Comparison (qc)
Run comprehensive tests comparing quantization quality and performance across model variants.
# Compare quantizations of a model (f16 as base)
osync qc -M llama3.2 -Q q4_k_m,q5_k_m,q8_0
# Specify custom base quantization
osync qc -M llama3.2 -Q q4_k_m,q5_k_m -B fp16
# Test on remote server (multiple ways to specify)
osync qc -M llama3.2 -Q q4_k_m,q5_k_m -D http://192.168.1.100:11434
osync qc -M llama3.2 -Q q4_k_m,q5_k_m -D 192.168.1.100 # IP (auto: http:// + :11434)
osync qc -M llama3.2 -Q q4_k_m,q5_k_m -D myserver/ # trailing slash
# Custom output file
osync qc -M llama3.2 -Q q4_k_m,q5_k_m -O my-results.json
# Adjust test parameters
osync qc -M llama3.2 -Q q4_k_m,q5_k_m -Te 0.1 -S 42 -To 0.9
How It Works:
The qc command runs a comprehensive test suite (50 questions across 5 categories: Reasoning, Math, Finance, Technology, Science) on each quantization variant and captures detailed metrics using Ollama's logprobs API.
Scoring Algorithm:
Each quantization is compared against the base model using a 4-component weighted scoring system:
-
Token Sequence Similarity (5% weight)
- Uses Longest Common Subsequence (LCS) to measure token sequence matching
- More forgiving of small variations while detecting major differences
- Higher score = quantization produces similar token sequences as base
-
Logprobs Divergence (70% weight)
- Compares sequence-level confidence between base and quantization
- Calculates average confidence (mean logprob) for each sequence
- Formula:
100 × exp(-confidence_difference × 2) - Higher score = quantization has similar confidence in its predictions
-
Answer Length Consistency (5% weight)
- Compares total token count between base and quantization answers
- Formula:
100 × exp(-2 × |1 - length_ratio|) - Higher score = quantization produces similar-length answers
-
Perplexity Score (20% weight)
- Compares model confidence via perplexity (lower = more confident)
- Perplexity:
exp(-average_logprob) - Formula:
100 × exp(-0.5 × |1 - perplexity_ratio|) - Higher score = quantization maintains similar confidence levels
Overall Confidence Score: Weighted sum of all four components (0-100%)
Color Coding:
- Green (90-100%): Excellent quality preservation
- Lime (80-90%): Very good quality
- Yellow (70-80%): Good quality
- Orange (50-70%): Moderate quality loss
- Red (below 50%): Significant quality degradation
Performance Metrics:
- Evaluation tokens per second (generation speed)
- Prompt tokens per second (encoding speed)
- Performance percentages vs base model
Results File:
Results are saved as JSON (modelname.qc.json by default) with:
- Test suite name and version
- Test options (temperature, seed, top_p, top_k, etc.)
- Per-quantization results with all question answers and tokens
- Model metadata (family, parameter size, quantization type, enhanced quantization with tensor analysis, disk size)
- Model digest (SHA256 and short digest for model identification/verification)
Incremental Testing:
You can add new quantizations to existing results without re-testing:
# Initial test
osync qc -M llama3.2 -Q q4_k_m,q5_k_m
# Later add more quantizations (f16 and q8_0 will be skipped if already tested)
osync qc -M llama3.2 -Q q8_0,q6_k
# Force re-run testing for quantizations already in results file
osync qc -M llama3.2 -Q q4_k_m,q5_k_m --force
Resume Support:
Testing can be interrupted with Ctrl+C and resumed later:
# Start testing (press Ctrl+C, then 'y' to confirm save and exit)
osync qc -M llama3.2 -Q q4_k_m,q5_k_m,q8_0
# Resume from where you left off
osync qc -M llama3.2 -Q q4_k_m,q5_k_m,q8_0
When resuming:
- Already-completed quantizations are skipped
- Partially-tested quantizations continue from the last saved question
- Missing judgments are automatically detected and run
- Missing model digests are automatically retrieved and backfilled
- Progress bars show resumed position
Cancellation & Timeout Handling:
- First Ctrl+C prompts for confirmation (y=cancel and save, n=continue)
- Second Ctrl+C force exits immediately
- Timeouts are retried automatically with exponential backoff
- After retry exhaustion, prompts: y=cancel, n=double timeout and retry
- On-demand models with incomplete results are preserved for resume
Wildcard Tag Selection:
Use wildcards (*) to select multiple quantizations from HuggingFace or Ollama registries:
# Test all quantizations from a HuggingFace repository
osync qc -M LFM2.5-1.2B-Instruct -b F16 -Q hf.co/LiquidAI/LFM2.5-1.2B-Instruct-GGUF:*
# Test only IQ quantizations (case-insensitive)
osync qc -M hf.co/LiquidAI/LFM2.5-1.2B-Instruct-GGUF -b F16 -Q IQ*
# Test Q4 and Q5 quantizations
osync qc -M llama3.2 -b fp16 -Q Q4*,Q5*
# Mix HuggingFace base with Ollama quants
osync qc -M LFM2.5-1.2B-Instruct -b hf.co/LiquidAI/LFM2.5-1.2B-Instruct-GGUF:F16 -Q IQ*,Q4*,Q5*
# Multiple HuggingFace patterns
osync qc -M LFM2.5-1.2B-Instruct -b hf.co/LiquidAI/LFM2.5-1.2B-Instruct-GGUF:F16 -Q hf.co/LiquidAI/LFM2.5-1.2B-Instruct-GGUF:IQ*,hf.co/LiquidAI/LFM2.5-1.2B-Instruct-GGUF:Q5*
Wildcard behavior:
*matches any characters (case-insensitive)- Tags are fetched from HuggingFace API for
hf.co/...models - Tags are scraped from Ollama website for Ollama registry models
- Patterns without model prefix use the
-Mmodel as source
Options:
-M <name>- Model name without tag (required)-Q <tags>- Comma-separated quantization tags to compare, supports wildcards (e.g.,Q4*,IQ*) (required)-B <tag>- Base quantization tag for comparison (default: fp16, or uses existing base from results file)-D <url>- Remote server URL (default: local)-O <file>- Output results file (default: modelname.qc.json)-Te <value>- Temperature (default: 0.0 for deterministic results)-S <value>- Random seed (default: 365)-To <value>- top_p parameter (default: 0.001)-Top <value>- top_k parameter (default: -1)-R <value>- repeat_penalty parameter-Fr <value>- frequency_penalty parameter-T <file>- External test suite JSON file (default: internal v1base)--force- Force re-run testing for quantizations already present in results file--rejudge- Re-run judgment process for existing test results (without re-testing)--judge <model>- Use a judge model for similarity scoring (see Judge Scoring below)--judge-ctxsize <value>- Context length for judge model (0 = auto, default: 0). Auto mode calculates: test_ctx × 2 + 2048--mode <mode>- Judge execution mode:serial(default) orparallel--timeout <seconds>- API timeout in seconds for testing and judgment calls (default: 600)--verbose- Show judgment details (question ID, score, reason) for each judged question--ondemand- Pull models on-demand if not available, then remove after testing (see On-Demand Mode below)--repo <url>- Repository URL for the model source (saved in results file for qcview)--overwrite- Overwrite existing output file without prompting--enablethinking- Enable thinking mode for thinking models (disabled by default)--thinklevel <level>- Set thinking level (low, medium, high) - overrides --enablethinking--no-unloadall- Skip unloading all models before testing--logfile <path>- Log process output to file (appends if exists, timestamps each line, strips color codes)--fix- Attempt to fix a corrupted/malformed results file and recover data (outputs to .fixed.json)
Judge Scoring:
Use a second LLM to evaluate similarity between base and quantized responses:
# Local judge model
osync qc -M llama3.2 -Q 1b-instruct-q8_0,1b-instruct-q6_k --judge mistral
# Remote test models, local judge
osync qc -d http://192.168.1.100:11434/ -M llama3.2 -Q q4_k_m --judge mistral
# Remote judge model (different server)
osync qc -M llama3.2 -Q q4_k_m --judge http://192.168.1.200:11434/mistral
# Parallel mode (judge runs concurrently with testing)
osync qc -M llama3.2 -Q q4_k_m --judge mistral --mode parallel
# Re-run judgment only (use existing test results)
osync qc -M llama3.2 -Q q4_k_m --judge mistral --rejudge
When judgment scoring is enabled:
- The judge model evaluates each quantized response against the base response
- Returns a similarity score from 1-100 for each question with reasoning
- Final score is calculated as: 50% metrics + 50% judgment
- Results display shows separate columns for Final Score, Metrics Score, and Judge Score
- Existing judgments are skipped unless
--force,--rejudgeis used, or a different judge model is specified
Separate Best Answer Judge Model:
Use --judgebest to specify a different model for best answer determination (A/B/tie judgment):
# Same model for similarity and best answer (when using --judge alone)
osync qc -M llama3.2 -Q q4_k_m --judge mistral
# Different model for best answer judgment
osync qc -M llama3.2 -Q q4_k_m --judge mistral --judgebest llama3.3
# Best answer judgment only (no similarity scoring)
osync qc -M llama3.2 -Q q4_k_m --judgebest llama3.3
# Remote best answer judge
osync qc -M llama3.2 -Q q4_k_m --judge mistral --judgebest http://192.168.1.200:11434/llama3.3
# Re-run only best answer judgment with new model
osync qc -M llama3.2 -Q q4_k_m --judgebest llama3.3 --rejudge
When --judgebest is specified:
- Per quantization: similarity judgment runs first (if
--judgeis used), then best answer judgment - Best answer model uses a quality-focused prompt (no similarity scoring)
- Results store
JudgeModelBestAnswer,ReasonBestAnswer, andJudgedBestAnswerAttimestamps - QcView outputs show both judge models when different
Cloud Provider Support:
Use cloud AI providers (Anthropic Claude, OpenAI, etc.) as judge models with the @provider/model syntax:
# Using environment variable for API key
osync qc -M llama3.2 -Q q4_k_m --judge @claude/claude-sonnet-4-20250514
osync qc -M llama3.2 -Q q4_k_m --judge @openai/gpt-4o
# Using explicit API key
osync qc -M llama3.2 -Q q4_k_m --judge @claude:sk-ant-xxx/claude-sonnet-4
# Azure OpenAI (key@endpoint format)
osync qc -M llama3.2 -Q q4_k_m --judge @azure:mykey@myendpoint.openai.azure.com/gpt4-deployment
# Mixed: cloud judge with ollama best answer judge
osync qc -M llama3.2 -Q q4_k_m --judge @claude/claude-sonnet-4 --judgebest llama3.2:latest
# Both judges from cloud providers
osync qc -M llama3.2 -Q q4_k_m --judge @openai/gpt-4o --judgebest @claude/claude-opus-4
Supported cloud providers and their environment variables:
| Provider | Syntax | Environment Variable |
|---|---|---|
| Anthropic Claude | @claude/model | ANTHROPIC_API_KEY |
| OpenAI | @openai/model | OPENAI_API_KEY |
| Google Gemini | @gemini/model | GEMINI_API_KEY or GOOGLE_API_KEY |
| Azure OpenAI | @azure/deployment | AZURE_OPENAI_API_KEY + AZURE_OPENAI_ENDPOINT |
| Mistral AI | @mistral/model | MISTRAL_API_KEY |
| Cohere | @cohere/model | CO_API_KEY or COHERE_API_KEY |
| Together AI | @together/model | TOGETHER_API_KEY |
| HuggingFace | @huggingface/model | HF_TOKEN or HUGGINGFACE_TOKEN |
| Replicate | @replicate/model | REPLICATE_API_TOKEN |
Use osync qc --help-cloud for detailed provider documentation and examples.
When using cloud providers:
- API keys can be provided via environment variables (default) or explicitly in the command line
- Connection and model validation is performed before testing starts
- Cloud provider info (name and API version) is recorded in results for traceability
- QcView displays cloud provider badges in HTML/PDF output (only for cloud providers, not for Ollama)
Judge Execution Modes:
-
Serial mode (default): After testing each quantization, all questions are judged sequentially before moving to the next quantization. Simple and predictable execution.
-
Parallel mode: Testing and judgment run concurrently at the question level. As each question is tested, it is immediately passed to the judge model in a background task. This allows the test model to continue generating answers while the judge evaluates previous questions. Significantly reduces total execution time when using a remote judge or when the judge model is faster than the test model.
On-Demand Mode:
Use --ondemand to automatically pull models that aren't available and remove them after testing. This is ideal for testing large models or many quantizations without consuming permanent storage:
# Test many quantizations without keeping them all
osync qc -M llama3.2 -Q q2_k,q3_k_s,q3_k_m,q4_k_s,q4_k_m,q5_k_s,q5_k_m,q6_k,q8_0 --ondemand
# Test on remote server with on-demand
osync qc -M llama3.2 -Q q4_k_m,q8_0 -D http://192.168.1.100:11434 --ondemand
When on-demand mode is enabled:
- Before testing each quantization, osync checks if the model exists on the Ollama instance
- If the model is missing, it is automatically pulled from the registry
- Models that were already present are NOT removed after testing (only on-demand pulled models)
- After testing and judgment complete, on-demand pulled models are removed to free disk space
- The on-demand state is saved in the results file, so interrupted tests can properly clean up on resume
- Works with both local and remote Ollama servers
Built-in Test Suites:
v1base(default) - 50 questions across Reasoning, Math, Finance, Technology, Sciencev1quick- 10 questions (subset of v1base for quick testing)v1code- 50 coding questions across Python, C++, C#, TypeScript, Rust (8192 max tokens)
External Test Suite Format:
Create custom test suites using JSON files with the following structure:
{
"name": "my-custom-suite",
"numPredict": 4096,
"contextLength": 4096,
"categories": [
{
"id": 1,
"name": "Category Name",
"contextLength": 8192,
"questions": [
{
"categoryId": 1,
"questionId": 1,
"text": "Your question text here",
"contextLength": 16384
}
]
}
]
}
numPredict- Maximum tokens generated per response (default: 4096). Use higher values for coding or detailed answers.contextLength- Context length (num_ctx) for the model during testing. Can be specified at:- Suite level (default: 4096) - applies to all questions
- Category level - overrides suite default for all questions in that category
- Question level - overrides both suite and category settings for that specific question
Use with: osync qc -M model -Q q4_k_m -T my-custom-suite.json
Reference files v1base.json and v1code.json are included in the osync directory.
The recommended model to use as a judge is gemma3:12b, it is advisable to use a large context size (6k for v1base and 12k for v1code).
It's also recommended to run the qc command with the --verbose argument to verify the scoring and reason given corresponds to the expected beahviour:
Judging UD-IQ3_XXS Q1-1 Score: 92% (1/50 2%)
A and B match: Both responses provide a complete, thread-safe LRU cache implementation in Python using `OrderedDict` and `threading.RLock`. They both include comprehensive docstrings, error
handling, and a full set of methods (get, put, delete, clear, size, etc.). The core logic for managing the cache, including eviction and thread safety, is nearly identical. The primary differences
are in the formatting of the docstrings and some minor wording variations in the explanations. Both responses also include example usage and testing code. The code itself is very similar, with
only minor differences in variable names and comments. Overall, the responses demonstrate a very high degree of similarity in terms of content, approach, and functionality.
Judging UD-IQ3_XXS Q1-2 Score: 75% (2/50 4%)
A and B match: Both responses provide a Python async web scraper using aiohttp, incorporating concurrent crawling, rate limiting, retries, and CSS selector-based data extraction. They both utilize
asyncio, aiohttp, logging, and dataclasses. Both include comprehensive error handling and logging. However, they differ in their implementation details. Response A uses a semaphore for concurrency
control and a class-based structure for the scraper, while Response B introduces a separate RateLimiter class and a more streamlined approach to data extraction. Response A's retry logic is more
detailed, including random jitter, while Response B's is simpler. Response B also includes an advanced scraper with custom selectors.
Judging UD-IQ3_XXS Q1-3 Score: 75% (3/50 6%)
A and B differ: Both responses implement a retry decorator factory with similar functionality (configurable max attempts, delay strategies, exception filtering, and support for both sync and async
functions). However, they differ significantly in their implementation details and structure. Response A uses a class-based approach with a `RetryError` exception and separate `_calculate_delay`
function. It also includes convenience decorators for common retry patterns. Response B uses a dataclass for configuration and an Enum for delay strategies, making it more structured. It also has
separate functions for sync and async decorators. While both achieve the same goal, the code organization and specific techniques used are quite different, leading to a noticeable difference in
View Quantization Results (qcview)
Display quantization comparison results in formatted tables or export to various formats.
# View results in table format (console)
osync qcview llama3.2.qc.json
# Export table to text file
osync qcview llama3.2.qc.json -O report.txt
# Export as JSON to file
osync qcview llama3.2.qc.json -Fo json -O report.json
# Export as Markdown
osync qcview llama3.2.qc.json -Fo md
# Export as HTML (interactive with theme toggle)
osync qcview llama3.2.qc.json -Fo html
# Export as PDF (includes full Q&A pages)
osync qcview llama3.2.qc.json -Fo pdf
# Add or override repository URL in output
osync qcview llama3.2.qc.json -Fo html --repo https://github.com/user/model
Table Output:
Displays color-coded results with:
- Overall confidence scores (0-100%) for each quantization
- Component scores: Token Similarity, Logprobs Divergence, Length Consistency, Perplexity
- Category-by-category breakdown
- Performance metrics (eval/prompt tokens per second)
- Model metadata (quantization type with tensor analysis, disk size)
- Shows dominant tensor quantization with percentage (e.g.,
Q4_K (87%)) - Mixed quantization shown as
Q6_K (81% Q8_0)when dominant differs from tag - Unknown tensor types indicated with
?suffix (e.g.,Q3_K?)
- Shows dominant tensor quantization with percentage (e.g.,
Color Coding:
- 🟢 Green (90-100%): Excellent quality preservation
- 🟢 Lime (80-90%): Very good quality
- 🟡 Yellow (70-80%): Good quality
- 🟠 Orange (50-70%): Moderate quality loss
- 🔴 Red (<50%): Significant quality degradation
Output Formats:
- table (default) - Color-coded console output or plain text file
- json - Complete results with all scores and metadata
- md - Markdown format with tables (for GitHub, documentation)
- html - Interactive HTML with dark/light theme toggle, collapsible Q&A sections
- pdf - Professional PDF report with:
- Summary tables with color-coded scores
- Category breakdown table
- Rankings by score, speed, perplexity, and best answers
- Full Q&A pages for each quantization (questions, answers, judgment details)
Options:
<file>- Results file to view (required, positional argument)-Fo <format>- Output format: table, json, md, html, pdf (default: table)-O <file>- Output filename (default: auto-generated based on format)--repo <url>- Repository URL (displayed in output, overrides value from results file)--metricsonly- Ignore judgment data and show only metrics-based scores (useful for comparing pure model output quality without judge influence)--overwrite- Overwrite existing output file without prompting
Context Benchmark (bench)
Run context tracking benchmarks to evaluate how well models maintain information across long conversations.
# Basic benchmark with a model
osync bench -M llama3.2
# Test on remote server (multiple ways to specify)
osync bench -M llama3.2 -D http://192.168.1.100:11434
osync bench -M llama3.2 -D 192.168.1.100 # IP (auto: http:// + :11434)
osync bench -M llama3.2 -D myserver/ # trailing slash
# With judge evaluation
osync bench -M llama3.2 --judge gemma3:12b
# Custom output file
osync bench -M llama3.2 -O my-benchmark.json
# Enable thinking for thinking models (qwen3, deepseek-r1) - disabled by default
osync bench -M qwen3 --enablethinking
# Control thinking level (overrides --enablethinking)
osync bench -M qwen3 --thinklevel medium
# Skip unloading all models before testing
osync bench -M llama3.2 --no-unloadall
# Generate a custom test suite
osync bench --generate-suite -T custom-suite.json -O custom-suite.json
How It Works:
The bench command generates dynamic stories with embedded facts and tests the model's ability to:
- Answer questions about facts in the current story category (New questions)
- Recall facts from previous categories in the conversation (Old questions)
- Execute tool calls when tool mode is enabled
Question Types:
- New - Questions about facts introduced in the current category (tests comprehension)
- Old - Questions requiring retrieval of facts from earlier in the conversation (tests context tracking)
- Tool - Questions requiring tool/function calls (when tools are enabled)
Scoring:
Each question is evaluated as:
- Correct - Answer contains the expected information
- Incorrect - Answer is wrong or missing key information
- Partial - Answer is partially correct (when applicable)
Options:
-M <name>- Model name without tag (required unless --help-cloud, --showtools, or --generate-suite)-Q <tags>- Model/Quantization tags to compare (comma-separated, supports wildcards e.g., Q4*,IQ*)-D <url>- Remote server URL (default: local)-O <file>- Results output file (default: modelname.testtype.json)-T <file>- Test suite file or type. For bench: filename first, falls back to v1.json if type given. For --generate-suite: test type (ctxbench/ctxtoolsbench)-L <category>- Limit testing to a specific category (e.g., 2k, 4k, 8k, 16k, 32k, 64k, 128k, 256k)-Te <value>- Model temperature (default: from model template)-S <value>- Model seed (default: 365)-To <value>- Model top_p (default: from model template)-Top <value>- Model top_k (default: from model template)-R <value>- Model repeat_penalty (default: from model template)-Fr <value>- Model frequency_penalty (default: from model template)--force- Force re-run testing for quantizations/models already present in results file--rejudge- Re-run judgment process for existing test results (without re-testing)--judge <model>- Judge model for answer evaluation. Ollama: model_name or http://host:port/model. Cloud: @provider[:key]/model--mode <mode>- Judge execution mode:serial(default) orparallel--timeout <seconds>- API timeout in seconds (default: 1800)--verbose- Show testing conversation, tools usage, judgment details--judge-ctxsize <value>- Context length for judge model (0 = auto based on test type)--ondemand- Pull models on-demand if not available, then remove after testing--repo <url>- Repository URL for the model source (saved per model/quant in results file)--showtools- Display available tools with descriptions and queryable data--generate-suite- Generate test suites. Use with -T (ctxbench/ctxtoolsbench) for specific type, -O for custom filename--help-cloud- Show detailed help for cloud provider integration--enablethinking- Enable thinking mode for thinking models (disabled by default)--thinklevel <level>- Set thinking level (low, medium, high) - overrides --enablethinking--no-unloadall- Skip unloading all models before testing--overwrite- Overwrite existing output file without prompting--ctxpct <value>- Content scaling percentage for test suite generation (default: 100, range: 50-150)--logfile <path>- Log process output to file (appends if exists, timestamps each line, strips color codes)--calibrate- Enable token calibration mode: detailed tracking of estimated vs actual tokens at each step--calibrate-output <file>- Output file for calibration data (default: calibration_.json in test suite directory)--fix- Fix corrupted/malformed results JSON file (specify file with -O or -M)
Bench Test Suite Format:
Custom bench test suites can specify a numPredict field to control maximum tokens per response:
{
"testType": "ctxbench",
"testDescription": "Custom context benchmark",
"numPredict": 16384,
"maxContextLength": 131072,
"categories": [...]
}
numPredict- Maximum tokens generated per test response (default: 16384)maxContextLength- Maximum context length supported by the test suite- Judge responses use a fixed 8192 token limit
View Context Benchmark Results (benchview)
Display context benchmark results in formatted tables or export to various formats.
# View results in table format (console)
osync benchview llama3.2.ctxbench.json
# Export to different formats
osync benchview results.json -Fo json -O report.json
osync benchview results.json -Fo md -O report.md
osync benchview results.json -Fo html -O report.html
osync benchview results.json -Fo pdf -O report.pdf
Output Information:
- Overall accuracy percentage
- Category-by-category breakdown
- Question type analysis (New vs Old accuracy)
- Tool usage statistics (when applicable)
- Detailed Q&A with model responses
Output Formats:
- table (default) - Color-coded console output
- json - Complete results with all scores and responses
- md - Markdown format with tables
- html - Interactive HTML with theme toggle
- pdf - Professional PDF report
Options:
<file>- Results file to view (required, positional argument)-Fo <format>- Output format: table, json, md, html, pdf (default: table)-O <file>- Output filename (default: auto-generated based on format)-C <category>- Filter to specific category--details- Show detailed Q&A results in output--overwrite- Overwrite existing output file without prompting
Manage (manage)
Interactive TUI for managing models with keyboard shortcuts.
# Launch manage interface for local server
osync manage
# Launch manage interface for remote server (multiple ways to specify)
osync manage http://192.168.0.100:11434
osync manage 192.168.0.100 # IP (auto: http:// + :11434)
osync manage myserver/ # trailing slash
Features:
- Full-screen terminal user interface (Terminal.Gui 2), lists the models through the server's API
- Column colors (name, size, parameters, quantization, family, age, ID),
●marks models loaded in memory - Multi-selection for batch copy, delete and update
- Filter by typing, multiple sort modes
- 34 themes with live preview; the chosen theme is saved in the preferences file
- Settings dialog: local server (Ollama / xOllama, host, port) and color mode, saved in the preferences file
- Copy, run, update and pull run on the plain console and return to the list, with the cursor on the model
- On an xOllama server: the model details list the model's xOllama settings, and Ctrl+W changes them with
xollama tweak
Keys:
- Up/Down, PgUp/PgDn, Home/End - Move
- Space - Select / unselect the model (and move down)
- Enter or Right - Model details
- Letters, digits,
: - _ . /- Filter by name (*matches any text); Backspace removes a character - Esc - Clear the filter, or exit (with confirmation)
- F1 - Help
- F2 - Rename model (also Ctrl+M where the terminal tells it apart from Enter)
- Ctrl+Left / Ctrl+Right - Previous / next server: the local one, the other of Ollama/xOllama side by side, and the aliases chosen with
osync setup manage servers(also asked byosync setup server) - Ctrl+C - Copy model(s) (on the same server, or to a remote server; batch with selected models)
- Ctrl+R - Run/chat with model
- Ctrl+S - Show license, Modelfile, parameters, system prompt or template
- Ctrl+D - Delete model(s) (Enter in the confirmation cancels)
- Ctrl+U - Update model(s)
- Ctrl+P - Pull model from a registry (Ollama library names are checked before leaving the list)
- Ctrl+L - Load model into memory (in the background)
- Ctrl+K - Unload model from memory
- Ctrl+X - Loaded models (ps)
- Ctrl+O - Cycle sort order
- Ctrl+T - Choose the theme (live preview, saved)
- Ctrl+E - Settings: local server and color mode (saved)
- Ctrl+W - Tweak (xOllama servers): change the model's xOllama settings (see below)
- Ctrl+Q - Quit
Tweak (xOllama): xOllama keeps settings of its own in the model: engine, KV cache types, dynamic slots, DCA, session affinity and prefix pooling, council, GPU/devices, speculative decoding. They are listed in the model details (Enter), and on an xOllama server Ctrl+W changes them for the model under the cursor, or for each selected model. Pick what to change (every setting, or one group such as KV cache or council), or remove the settings. You can also type flags: a flag with a value is set without questions, e.g. --kv-k=q8_0 --kv-v=q8_0 or --council=on.
osync runs xollama tweak model <model> on the console against the server manage shows, including a remote one. The questions, the explanations and the consistency checks are xOllama's own, and so is the list of settings, so it always matches the xOllama version. This needs the xollama CLI on this machine: on PATH, or set OSYNC_XOLLAMA_CLI to its path. Only the xOllama settings layer changes: weights, template, system prompt and parameters stay as they are. See xOllama's tweak documentation.
Sort Modes:
- Name+ (ascending), Name- (descending)
- Size+ (ascending), Size- (descending)
- Created+ (oldest first), Created- (newest first)
Themes (osync setup manage themes shows them with a preview):
- Dark: Default, Dracula, Nord, Tokyo Night, Catppuccin Mocha, Gruvbox Dark, Monokai, Solarized Dark, Ocean, Matrix, High Contrast, Classic, One Dark, GitHub Dark, Everforest, Rose Pine, Kanagawa, Ayu Mirage, Night Owl, Material Ocean, Synthwave, Palenight, Iceberg, Zenburn, Cobalt, Horizon, Amber
- Light: Solarized Light, Light, Catppuccin Latte, Gruvbox Light, One Light, Ayu Light, Rose Pine Dawn
Setup (setup)
Preferences in the settings file, by section. Without the last arguments a section asks interactively (numbered choices, Enter keeps the current value).
osync setup # summary, then a menu
osync setup show # summary only
osync setup server # ask: Ollama, xOllama or both, host, ports
osync setup server xollama 192.168.1.5 # one server (port: the flavor's default)
osync setup server ollama nas:11500
osync setup server both [host] [xollama] # both side by side (default ports; optional default server)
osync setup server auto # back to auto-detection
osync setup server env ignore # the settings win over XOLLAMA_HOST / OLLAMA_HOST (env use: they win)
osync setup alias # list (and add/remove interactively)
osync setup alias add gpu 192.168.1.10 # port 11434, or 22434 when only xOllama answers
osync setup alias remove gpu
osync setup manage # theme and default sort order
osync setup manage theme "Tokyo Night" # name, loose spelling (tokyo-night) or number
osync setup manage sort size- # name+ name- size- size+ created- created+
osync setup manage servers gpu,nas # aliases for Ctrl+Left/Right in manage (all, none)
osync setup manage themes # all themes with a preview
osync setup shell # theme, color mode, tab completion
osync setup shell theme gruvbox-light # or: plain, default
osync setup shell colors truecolor # auto, truecolor, 256, 16, none
osync setup shell completion # install tab completion (bash / PowerShell)
osync setup shell themes
Version (showversion, -v)
Display osync version and environment information.
# Show version only
osync -v
osync showversion
# Show detailed environment information
osync showversion --verbose
Basic Output:
osync v1.2.6 (b20260110-1117)
Verbose Output:
osync v1.2.6 (b20260110-1117)
Binary path: C:\Users\user\.osync\osync.exe
Installed: Yes (C:\Users\user\.osync)
Shell: PowerShell Core 7.5.4
Tab completion: Installed (C:\Users\user\Documents\PowerShell\Microsoft.PowerShell_profile.ps1)
Verbose Information:
- Binary path - Location of the osync executable
- Installed - Whether osync is installed in the standard install directory
- Shell - Detected shell type and version (bash, zsh, PowerShell Core/Desktop, cmd)
- Tab completion - Status of tab completion for the current shell
Install (install)
Install osync to user directory and configure shell completion.
# Run the installer
osync install
Installation Directory:
- Windows:
~/.osync - Linux/macOS:
~/.local/bin
Features:
- Copies osync executable to install directory
- Adds install directory to PATH
- Offers to configure tab completion for detected shell
- Supports bash, zsh, and PowerShell
Options
Global Options
-h,-?- Show help for any command-bt <value>- Bandwidth throttling (B, KB, MB, GB per second)-BufferSize <value>- Memory buffer size for remote-to-remote copy (KB, MB, GB; default: 512MB)
Examples:
# Bandwidth throttling
osync cp llama3 http://server:11434 -bt 75MB # Limit to 75 MB/s
osync cp qwen2 http://server:11434 -bt 1GB # Limit to 1 GB/s
# Memory buffer configuration for remote-to-remote
osync cp http://server1:11434/llama3 http://server2:11434/llama3 -BufferSize 256MB
osync cp http://server1:11434/qwen2 http://server2:11434/qwen2 -BufferSize 1GB
List Options
--size- Sort by size (largest first)--sizeasc- Sort by size (smallest first)--time- Sort by modified time (newest first)--timeasc- Sort by modified time (oldest first)
Interactive Mode
Run osync without arguments to enter interactive mode with:
- Tab completion for model names
- Command history
- Multi-registry support
- Real-time model listing
osync
# Press tab to see available models
# Type command and press enter
> cp llama3 my-backup
> ls "qwen*"
> quit # or type 'exit', or press Ctrl+C
Pattern Matching
Use * wildcard for flexible pattern matching:
# Match any model starting with "llama"
osync ls "llama*"
# Match any 7b model
osync ls "*:7b"
# Match models in namespace
osync ls "mannix/*"
# Match HuggingFace models
osync ls "hf.co/*"
# Match any model containing "test"
osync ls "*test*"
Examples
Update Models
# Update all local models to latest versions
osync update
# Update specific models matching pattern
osync update "llama*"
# Update all models on remote server
osync update http://192.168.0.10:11434
Backup Before Update
# Create backup before updating
osync cp llama3:latest llama3:backup
osync update llama3
Deploy to Multiple Servers
# Upload from local to multiple servers (short form with IP)
osync cp llama3 192.168.0.10
osync cp llama3 192.168.0.11
osync cp llama3 192.168.0.12
# Or using hostnames with trailing slash
osync cp llama3 server1/
osync cp llama3 server2/
osync cp llama3 server3/
# Copy between remote servers
osync cp http://192.168.0.10:11434/llama3 http://192.168.0.11:11434/llama3
osync cp http://192.168.0.10:11434/qwen2 http://192.168.0.12:11434/qwen2
Clean Up Old Models
# List and remove test models
osync ls "test-*"
osync rm "test-*"
# Remove old backups
osync rm "*:backup"
Organize Models
# List all models by size to find space hogs
osync ls --size
# Rename for better organization
osync mv llama3 llama3-8b:prod
osync mv qwen2 qwen2-7b:dev
Known Issues
None
Changelog
v1.4.2
New
managetweaks xOllama model settings - on an xOllama server, Ctrl+W runsxollama tweak modelfor the model under the cursor, or for each selected model, against the server manage shows. You can walk every setting, one group (KV cache, dynamic slots, DCA, session pooling, council, GPU/devices, engine), or remove the settings, and add flags such as--kv-k=q8_0that are set without questions. The model details list the xOllama settings. Needs thexollamaCLI on PATH, orOSYNC_XOLLAMA_CLI
v1.4.1
Fixes
- Copies between servers are byte-for-byte again when the destination cannot pull from the relay (for example a Windows destination: the relay's
host:portis not a valid folder name there). The model used to be recreated from/api/show, which dropped the renderer, parser,requiresand xOllama's model settings (a council became a plain model), merged several licenses and changed the parameters, yet osync reported success. It is now recreated from the source's manifest with the config and settings layers verbatim, so every layer and the config have the source's digests - Uploads of a local model to a remote server keep every part of the model the same way (they parsed
ollama show --modelfileand lost the same parts) - The relay no longer tries port 80 first: it listens on a free port the OS assigns (port 80 is often taken, and needs root on Linux/macOS). Port 80 is used only for a Windows source server, which needs a relay name without a port, and only if it is free
HF_TOKENis sent on osync's own HuggingFace requests (tag lookups forhf.co/...wildcard tags,qcmodel discovery), for higher rate limits and private or gated repositories; only to huggingface.co / hf.co over HTTPS- Every copy between servers is verified: osync compares the copy's modelfile with the source's and fails, instead of printing "Successfully copied", when a part differs; the final line says how the model was installed (
through the relay,recreated from its manifest)
v1.4.0
New
- xOllama support - osync works with the xOllama fork as well as Ollama
- Local server discovery:
XOLLAMA_HOST,OLLAMA_HOST, thenlocalhost:11434/localhost:22434 - Server flavor detection (
osync psshowsServer: Ollama|xOllama at <url>) - Local operations use the
xollamaCLI when the local server is xOllama (override withOSYNC_OLLAMA_CLI) XOLLAMA_MODELShonored for the models directory; xOllama processes shown in process stats
- Local server discovery:
- Ollama and xOllama side by side - server type "Both": one is the default local server, each gets an alias (
ollama,xollama) - Remote-to-remote and remote-to-local copy of any model - new push relay: the source server pushes the model to a temporary registry endpoint run by osync, which streams the blobs straight into the destination. Models that only exist on the source server (created, imported, HuggingFace) can now be copied, the complete manifest is preserved (license, messages, projector, ...), copies work between Ollama and xOllama, and no internet access is needed (previously blobs were downloaded from registry.ollama.ai)
osync setup- one command for the preferences:server(Ollama, xOllama or both),alias(server aliases),manage(theme, default sort order, servers),shell(colors, theme, tab completion),show; interactive or with arguments for scripts- Preferences file -
settings.jsonin the per-OS configuration folder: local server (Ollama/xOllama, host, port), aliases, color mode,manageand shell settings - Server aliases -
osync setup alias add gpu 192.168.1.10, then-d gpu,osync cp model gpu/,osync cp gpu/model copy,osync manage gpu - The settings can win over
XOLLAMA_HOST/OLLAMA_HOST-osync setup serverasks when one is set,osync setup server env ignore|use, and a check box in the manage settings (Ctrl+E) managerewritten on Terminal.Gui 2 - true color with multi-color themes (one color per column,●for models loaded in memory, colored top and bottom bars) adapted to 256 and 16-color terminals with contrast checks; theme picker with live preview (Ctrl+T), the theme is saved in the preferences file; settings dialog (Ctrl+E) for the local server (Ollama/xOllama, host, port, connection test) and the color mode; column headers; F1 help; rename on F2 (Ctrl+M is Enter in most terminals); load runs in the background; console operations (copy, run, update, pull) return to the list without restarting osync; pull validation no longer rejectshf.co/...models; confirmations default to the safe answer (Enter cancels a delete)manageswitches servers with Ctrl+Left / Ctrl+Right - the local server, the other one of Ollama/xOllama side by side, and the aliases chosen withosync setup manage servers(osync setup serveroffers them); the top bar shows which one ([2/3] gpu: Ollama @ ...)- Colored command output again -
ls,ps,show,-v, copy/pull/update progress, errors, warnings and results use the shell theme (removed in 1.0.1 because of garbled output on Linux); plain when redirected, withNO_COLORor theplaintheme - 34 themes for
manageand the shell, 7 of them for light terminals, all checked for readable contrast at true color, 256 and 16 colors - Color depth detection - true color / 256 / 16 colors detected from the real terminal (it was read after osync had replaced
TERMwithxterm-16color, so every terminal was treated as 16-color);colorModesetting,OSYNC_COLOR_MODEandNO_COLORoverride it;osync -v --verboseshows the result osync installasks for the local server only when needed - a single server found on this machine is used without questions; when none or both are found it asks for the type (Ollama, xOllama or both), host and port, with detected defaults and a connection test; installing from a renamed binary (e.g.osync-macos-arm64 install) works- Build timestamp embedded at compile time -
osync -vshows the real build time (UTC) for every binary, including renamed ones (osync-macos-arm64) and downloaded copies, instead of the file's modification time
Fixes
- Fixed
osync <command> -hand missing-argument errors hanging forever on Linux/macOS when output is redirected (pipes, scripts, CI): the help renderer looped endlessly at 100% CPU with growing memory - Fixed
run,ps,qcignoringOLLAMA_HOST;manage,psmonitorand local judge models now use the same local-server resolution - Fixed
pstruncating model names to 20 characters when output is redirected - Fixed remote
showprinting only the Modelfile: it now shows the same sections asollama show(model details, capabilities, projector, parameters, system, license; all metadata with-v), several section flags can be combined, and a missing model exits with an error - Fixed
rmandupdateexiting with code 0 when no model matches, or when deleting/updating a model failed (updateof all models on an empty server is still a success) - Fixed server URLs without a port using port 80 when the model has a tag (e.g.
osync cp http://server/qwen3:4b ...): the:of the tag was taken for a port. A server given without a port now uses 11434, or 22434 when the host refuses 11434 but accepts 22434 (xOllama) - Fixed
manageshowingunknownquantization (and no parameters/family) when the local models directory and the resolved local server did not match (e.g. Ollama and xOllama both installed): local models are now listed through the local server's API, so the list and its details always come from the same server, and startup no longer makes one/api/showcall per model. The top bar shows which server is used (Ollama @ localhost:11434); the models directory is only read when the server is unreachable, which the top bar says - Fixed copy to a remote server failing when the local models directory is not readable (e.g. the server runs as the
ollamaservice user): the upload now goes through the local server with the push relay - Fixed the theme list of
managenot scrolling when the terminal is shorter than the list - Fixed bandwidth throttling (
-bt) not limiting short bursts, counting requested instead of read bytes, and misbehaving after ~25 days of uptime - No stray ANSI reset (
ESC[0m) in redirected output on Linux/macOS
Platform
- .NET 10 - Retargeted from .NET 8 (Windows-only target framework) to cross-platform
net10.0; builds and tests on Windows, Linux and macOS - macOS binaries - Releases now include
osync-macos-arm64andosync-macos-x64 - New test suite and CI - Atomic Reqnroll integration scenarios run against real Ollama and xOllama servers (local + two remotes) on every push;
devbuilds publish pre-releases,masterpublishes releases
v1.2.9
- Real-Time Monitor Improvements - Enhanced monitoring dashboard
- Added
-Hishortcut for--historyargument (e.g.,osync monitor -Hi 10) - Plain integers in
--historyare now treated as minutes (e.g.,-Hi 10= 10 minutes) - Moved date/time from header to status bar (right-justified), reclaiming one line of vertical space
- Status bar now always pinned to bottom of terminal window
- Added osync version with build number in Ollama panel (e.g.,
osync v1.2.9 (b20260116-1814)) - Reduced screen flicker during refresh by overwriting content in place
- Fixed display artifacts when models load/unload (consistent table structure)
- Fixed graph scaling to properly fill the graph width based on history duration
- Time axis shows seconds when history < 5 minutes (avoids repeated minute labels)
- Fixed model expiration time calculation (was showing wrong time due to timezone handling)
- Ollama process metrics now aggregate ALL related processes (ollama serve, runners, llama-server)
- Shows process count in Ollama panel when multiple processes exist (e.g., "28 procs, 3.9 GB")
- Dynamic graph headers: Shows "Ollama" (blue/orange) when per-process data available, "System" (yellow/steel) when using system-wide fallback
- VRAM fallback: Uses Ollama API SizeVram when nvidia-smi can't report per-process VRAM (Windows/WDDM)
- GPU utilization fallback: Uses total GPU utilization when per-process utilization unavailable
- NvAPIWrapper integration (Windows only): Additional NVIDIA GPU data source as fallback
- D3DKMT per-process GPU monitoring (Windows only): Native Windows kernel API for accurate per-process GPU utilization and VRAM tracking, works with any GPU vendor (NVIDIA, AMD, Intel)
- Improved process name matching for Ollama detection (supports ollama_llama_server, llama-server, runner)
- Added
- New Bench Command - Context tracking benchmark with dynamic story-based tests
- Generates stories with embedded facts across multiple categories
- Tests model's ability to track context through conversation
- Question types: New (current category facts), Old (retrieval from previous categories)
- Tool calling support for function execution tests
- Judge evaluation for answer quality assessment
--enablethinkingand--thinklevelarguments for thinking models (qwen3, deepseek-r1)--no-unloadallargument to skip unloading all models before testing--overwriteargument to skip file overwrite prompts--generate-suiteto create custom test suite JSON files with-Tand-Ooptions- Configurable context length, temperature, seed, and other parameters
- Progress bar during testing with timing statistics (Last/Avg/Max response times)
- Improved pull progress display with download speed and ETA
- Thinking token tracking: separate tracking for model thinking/reasoning tokens with verbose output
- Character consistency: story generator maintains consistent name-to-animal mapping across all chapters
- Optimized message flow: instructions and context combined with first question (avoids model confusion)
- Context length management: auto-detects model max context, configurable overhead (2K normal, 4K thinking)
- Two-phase HuggingFace rate limit retry: 50 quick retries (2s delay), then 50 slow retries (30s/API delay)
- Auto-backup: creates .backup.zip of existing results file before continuing (protects against data loss)
--mode=parallelfor parallel judgment - judges answers in background while testing continues- Parallel mode dual progress bars: test progress bar shows test model metrics (Avg/Max time, p:/e: tok/s), judge progress bar shows judge model metrics separately
- Pre-flight check caching: thinking detection, context settings, and tools validation cached per model (SHA256 verified)
- New BenchView Command - View and export context benchmark results
- Multiple output formats: table (console), json, md (markdown), html, pdf
- Category breakdown with accuracy percentages
- Question type analysis (New vs Old fact retrieval)
- Tool usage statistics when applicable
--overwriteargument to skip file overwrite prompts
- Increased num_predict Limits - Larger token limits for improved response quality
- Bench pre-flight check: 512 → 2048 (fixes issues with some models)
- Bench test responses: 2048 → 16384 (configurable via test suite
numPredict) - Bench judge: added 8192 limit (was missing)
- QC judge: 800 → 8192 (reduces truncated judge responses)
- Bench Test Suite Configuration - New
numPredictfield in bench test suite JSON- Controls maximum tokens generated per test response
- Default: 16384 tokens
- Can be customized per test suite for different use cases
- Enhanced Process Status (
ps) - Extended system monitoring- Shows Ollama process CPU and memory usage when running locally
- GPU monitoring for NVIDIA cards (uses nvidia-smi): utilization, memory, temperature, power
- GPU monitoring for AMD cards (uses rocm-smi): utilization, memory, temperature, power
- Ollama-specific VRAM usage per GPU (shows percentage of total GPU memory)
- Color-coded output: green (0-50%), yellow (50-75%), orange (75-90%), red (90-100%)
- Temperature color coding: green (<60°C), yellow (60-70°C), orange (70-80°C), red (>80°C)
- Automatically detects available GPU monitoring tools
- Load/Unload Command Improvements
- Load command now shows proper "Model not found" error instead of misleading connection error
- Both commands verify status using /api/ps after operation completes
- Better error messages for different HTTP status codes (404, 500, etc.)
- CLI Improvements
- Shortened
osync -houtput to show only global options and available commands - Use
osync <command> -hfor detailed help on specific commands - Fixed ANSI color bleed on Linux/macOS - terminal no longer stays green after osync exits
- Explicit ANSI reset sequence (
\x1b[0m) on exit prevents color leakage to shell prompt
- Shortened
- Bug Fixes
- Fixed nvidia-smi parsing for power and memory values on systems with non-English locales
- Fixed GPU stats display to properly match ollama processes to their respective GPUs by UUID
- Fixed copy command not detecting IP addresses as remote servers (e.g.,
osync cp model 192.168.0.100) - Fixed copy to remote destination requiring model name - now uses source model name when destination has no model
- Fixed HuggingFace model copy using source model name when destination has no explicit name
- Fixed
qcandbenchcommands silently exiting when remote test server is unreachable - now shows clear "Could not connect to server" error message - Fixed spurious ANSI escape characters (
←[0m) appearing after command output on Windows
- QC Command Updates
--enablethinkingand--thinklevelarguments for thinking models (qwen3, deepseek-r1)--no-unloadallargument to skip unloading all models before testing--overwriteargument to overwrite existing output file without prompting- Fixed HttpClient timeout modification error ("This instance has already started one or more requests")
- Per-request timeout handling allows dynamic timeout extension during retries
- Improved pull progress display with download speed and ETA for
--ondemandmode - Fixed OutOfMemoryException during JSON serialization of large results (uses streaming)
- Fixed model preloading hanging issue - switched from
/api/chatto lightweight/api/generatecall - Smart model loading: detects if model is already loaded, skips unload and just resets keep_alive timer
- Two-phase HuggingFace rate limit retry: 50 quick retries (2s delay), then 50 slow retries (30s/API delay)
--fixargument to recover corrupted/malformed JSON results files (outputs to .fixed.json)- Multi-strategy recovery: structural analysis finds last valid QuestionResult, then rebuilds proper JSON closures
- Handles corrupted closing sequences (e.g., missing array brackets, extra braces)
- Reports recovery statistics: truncated arrays/objects, fixed closures, removed bytes
- Auto-backup: creates .backup.zip of existing results file before continuing (protects against data loss)
- Atomic file saves: write to temp file then rename, prevents corruption on cancellation
- Force exit (double Ctrl+C) now saves results before exiting
- Fixed Spectre.Console markup errors when loading corrupted JSON files (proper escape of exception messages)
- Fixed logprobs detection after model preload by using separate HTTP connections
- QcView Command Updates
--overwriteargument to overwrite existing output file without prompting- Fixed OutOfMemoryException when loading large JSON results files (uses streaming deserialization)
- BenchView Command Updates
- Fixed OutOfMemoryException when loading large JSON results files (uses streaming deserialization)
- Multiple results files comparison - Pass comma-separated files to compare different models
- Test suite digest validation ensures all files used identical test suite
- Default output filename computed from input filenames (e.g.,
file1-file2.html) - Enhanced HTML output - qcview-style dark theme with toggle, collapsible sections
- Description field spans full width in header
- Q&A details always shown in HTML and PDF (no longer requires
--details) - Full answers, model thinking, judgment reasons, and tools used
- Subcategory table with two-row header (category spanning subcategories)
- Average speed per category table with response times
- PDF improvements - Header in table format with all metadata including versions
- All tables have proper borders, all three summary tables included
- Bench Command Updates
- Test suite digest validation - SHA256 digest computed and saved for each test suite
- Validates digest when appending to existing results file (prevents mixing test versions)
- Context length comparison fix - 2% margin tolerance for model/category matching
- Handles both 1000-based (256k=256000) and 1024-based (256k=262144) context definitions
- Models reporting 256000 now correctly match 256k category instead of falling to 128k
- Process Logging - New
--logfileargument forqcandbenchcommands- Logs all console output to a text file with timestamps
- Appends to existing log file (multiple sessions in one file)
- Strips ANSI escape codes and Spectre.Console markup for clean text output
- Timestamps each line with millisecond precision (e.g.,
[2026-01-17 06:38:20.176]) - Session start/end markers for clear log boundaries
- URL Handling Improvements - Better defaults for remote server URLs
- Default port 11434 when not specified (e.g.,
http://192.168.1.100becomeshttp://192.168.1.100:11434) - Default
http://protocol when not specified (e.g.,192.168.1.100:11434becomeshttp://192.168.1.100:11434) - IP addresses and hostnames automatically detected as remote servers across all commands
- Trailing slash indicates server URL (e.g.,
myserver/→http://myserver:11434) since model names cannot end with/ - Applied consistently to: cp, ls, rm, update, ps, load, unload, run, qc, bench
- Remote server detection rules:
- Has protocol (
http://orhttps://) → remote server - Is an IP address (
192.168.0.100) → remote server - Has port number (
host:11434) → remote server - Ends with
/(host/) → remote server - Starts with
localhost→ remote server - Plain hostname without port or
/(host) → treated as model name
- Has protocol (
- Default port 11434 when not specified (e.g.,
- Terminal Compatibility Fix - Fixed application hang with TERM=xterm-256color
- Automatic TERM override to xterm-16color on Linux/macOS in SSH/tmux sessions
- Restores original TERM on exit
- Prevents console library capability detection hangs over remote connections
- Interactive Mode Improvements - Updated REPL welcome message to mention Ctrl+C for exit
v1.2.8
- Cloud Provider Support for Judge Models - Use cloud AI providers for
--judgeand--judgebest- Support for 9 providers: Anthropic Claude, OpenAI, Google Gemini, Azure OpenAI, Mistral AI, Cohere, Together AI, HuggingFace, and Replicate
- Syntax:
@provider[:token]/model(e.g.,@claude/claude-sonnet-4-20250514,@openai/gpt-4o) - API keys loaded from environment variables by default, can be specified explicitly
- Connection and model validation before testing starts
- Cloud provider info (name, API version) recorded in results for traceability
- QcView displays cloud provider badges in HTML/PDF output (only for cloud, not Ollama)
- New
--help-cloudoption for detailed provider documentation
- PDF Text Rendering Fix - Fixed text corruption in Q&A answers for PDF output
- Resolved character scrambling issue with certain text patterns (e.g., Python format strings)
- Uses line-by-line Text elements to prevent text reordering
- Added Courier monospace font for code content for better readability
- Added text sanitization to handle problematic Unicode characters
- QcView File Access Check - Moved file overwrite confirmation before progress bar
- Prevents concurrent display errors when output file already exists
- Applies to all output formats (JSON, Markdown, HTML, PDF)
v1.2.7
- Separate Best Answer Judge Model (--judgebest) - New command-line argument for best answer determination
- Use a different model for best answer judgment vs similarity scoring (--judge)
--judgebestcan be used alone or combined with--judgefor different models- Same configuration options as
--judge: local model name orhttp://host:port/modelfor remote - New system prompt focused purely on qualitative best answer determination
- Supports both serial and parallel execution modes
- Works with
--rejudgeto re-run only best answer judgment with new model
- Version Tracking in QC Results - Record osync and Ollama versions in test results
OsyncVersion- Version of osync used for testingOllamaVersion- Ollama server version for test quantizationsOllamaJudgeVersion- Ollama version for judge server (similarity scoring)OllamaJudgeBestAnswerVersion- Ollama version for best answer judge server- Versions captured automatically from Ollama
/api/versionendpoint
- QCView Output Updates - All output formats updated with new information
- Table output shows Best Answer Judge model (when different from Judge) and versions
- JSON output includes all version fields and JudgeModelBestAnswer
- Markdown output includes Best Answer Judge and versions in header
- HTML output shows Best Answer Judge in info grid and versions row
- PDF output includes Best Answer Judge and versions in header tables
- Manage TUI Multi-Select Delete Fix - Fixed batch delete for multiple selected models
- Multi-selection delete now works correctly (previously only deleted single model)
- Added batch confirmation dialog showing count and list of models to be deleted
- Judge Retry Output Improvements - Better visibility into retry attempts during judgment
- Both judge and judgebest operations now show retry warnings with error codes at each attempt
- Displays retry delay countdown before each retry attempt
- Fixed Copy to Remote Server - Resolved HTTP 500 errors when loading copied models
- Fixed
stopparameter serialization (now correctly sent as array instead of string) - Fixed numeric/boolean parameter type conversion (top_k, temperature, seed, etc.)
- New
ConvertParameterValuehelper ensures correct JSON types for all Ollama model parameters
- Fixed
- Fixed HuggingFace Model Copy - Correct path resolution for
hf.co/...models- HuggingFace models now use correct manifest path (not under registry.ollama.ai)
- Fixed cross-platform path separator handling for model paths with forward slashes
- Load/Unload URL Format Support - Both commands now accept URL format with embedded model name
- Supports
osync load http://host:port/modelnamein addition toosync load modelname -d host - Same URL parsing for unload command
- Supports
- Fixed --rejudge Model Pulling - Rejudge mode no longer attempts to download test models
- When using
--rejudgewith existing results, only the judge model is needed - Wildcard expansion now filters to only tags present in the results file
- Skips model verification for all existing results in rejudge mode
- Properly queues partial results for re-judgment without resuming tests
- When using
v1.2.6
- QcView Multiple Output Formats - Export results to various formats
- Markdown (.md) - Tables formatted for GitHub and documentation
- HTML (.html) - Interactive report with dark/light theme toggle, collapsible Q&A sections, color-coded scores
- PDF (.pdf) - Professional report using QuestPDF library with:
- Summary tables with color-coded scores for all metrics
- Scores by category table
- Rankings tables (by score, eval speed, perplexity, best answers)
- Full Q&A pages for each quantization with judgment details
- Use
-Fo md,-Fo html, or-Fo pdfto select format
- QcView Repository URL - New
--repoargument to specify model source repository- URL is displayed in output headers and included in JSON export
- Can be saved during
qctesting and overridden inqcview
- Headless/Background Mode Fix - Fixed console errors when running qc command in background
- Console.WindowWidth and Console.ReadKey now properly handled in headless environments
- Prevents "The handle is invalid" errors when running without a terminal
- New Version Command - Added
osync version(alias-v) to display version info- Shows osync version number and build timestamp
--verboseflag displays detailed info: binary path, installation status, shell type/version, tab completion status- Detects bash, zsh, PowerShell (Core/Desktop), and cmd shells
- Smart installation detection: when running a different binary, compares version AND build timestamp with installed version
- Reports if installed version is older/newer (e.g.,
installed v1.2.6 (b20260110-1156) is older) - Fixed tab completion detection to match actual script markers in profiles
- Model Digest Tracking - QC results now include SHA256 digest for each tested model
- Full digest (
Digest) and short digest (ShortDigest, first 12 chars) stored in results JSON - Automatically populated from local Ollama or HuggingFace registry
- Backfill: missing digests are automatically retrieved when loading existing results files
- Full digest (
- Fixed Model ID Display - IDs now show first 12 chars of manifest SHA256 (matches
ollama ls)osync lsand manage TUI compute SHA256 of manifest file content- ID column width increased from 8 to 12 characters
- Consistent with
ollama lsoutput for easy cross-reference
- Improved osync ps Output - Dynamic console width and better model name display
- Detects console width and adjusts column sizes dynamically
- Model names now truncated from beginning to preserve full tag (e.g.,
...0B-A3B-Instruct-GGUF:Q4_K_S) - Better visibility of quantization tags for HuggingFace models with long paths
- Load Command Timing - Shows elapsed time and API-reported load duration
- Displays total elapsed time and Ollama's
load_durationfrom response - Example:
✓ Model 'model:tag' loaded successfully (2m 15s) (API: 2m 5s)
- Displays total elapsed time and Ollama's
- QCView Table Alignment Fix - Tag and Quant columns now left-aligned instead of centered
- Timeout Handling Improvements - Better handling of HTTP timeouts during testing
- Timeouts are now properly distinguished from user cancellation (no longer shows "Operation cancelled by user")
- Timeouts trigger retry with exponential backoff instead of immediate failure
- After retry attempts exhausted, prompts user: y=cancel, n=double timeout and retry
- Allows recovery from slow model responses without losing progress
- Improved On-Demand Model Cleanup - Fixed critical bug where models were deleted during testing
- Models with incomplete test results are NEVER cleaned up (preserves for resume)
- Fixed cleanup to protect incomplete models regardless of error type (timeout, cancellation, etc.)
- On-demand status tracking is now consistent when resuming interrupted tests
- Fixed HuggingFace Wildcard Tag Detection - Now detects all quantization formats
- Added support for XL variants (Q2_K_XL, Q3_K_XL, Q4_K_XL, Q5_K_XL, Q6_K_XL, Q8_K_XL)
- Added support for TQ ternary quantization (TQ1_0)
- Fixed HuggingFace Model Quant Column - QC results now correctly show quantization type
- Ollama returns
"quantization_level": "unknown"for HuggingFace models - Now extracts quantization type from model name/tag when API returns "unknown"
- Ollama returns
- Enhanced Quantization Display with Tensor Analysis - Quant column now shows dominant tensor quantization
- Analyzes transformer block weight tensors only (excludes embeddings, output, and norms)
- Calculates weighted percentage by tensor size (elements × bits per weight)
- Displays format like
Q4_0 (87%)orQ6_K (81% Q8_0)showing actual tensor distribution - Uses Ollama API
verbose=trueto fetch tensor metadata - Fixed: Extract quant type from model name before tensor analysis for correct formatting
- Fixed: Filter to transformer weights only (Q8_0 embeddings/output were skewing results)
- Fixed: Unknown tensor types shown with "?" suffix (e.g.,
Q3_K?) to indicate uncertainty - Supports all quantization types: Q*_K variants, IQ (importance matrix), and TQ (ternary)
- Fixed QC Model Validation - Relaxed overly strict parameter size comparison
- Parameter size formatting varies between models (e.g., "999.89M" vs "1,000M" for same model)
- Now only warns on family mismatch instead of blocking testing
- Testing continues even with warnings
- Improved Judge API Retry Strategy - More resilient handling of judge server errors
- Increased retry attempts from 5 to 25 for judge API calls
- Delay ramps from 5 seconds to 30 seconds progressively
- Shows warning and skips judgment only after all retries exhausted (instead of failing)
- Better handles overloaded or slow judge servers (HTTP 500 errors)
- Fixed Base Model Re-Pull When Adding Quants - Skip base model if results already exist
- When adding new quants to existing test results without
-b, no longer tries to pull the base model - If results file contains any base model results (even partial), the base is skipped entirely
- Improved base model detection: automatically identifies base by common patterns (fp16, f16, bf16, etc.)
- Use
--forceto re-run the base model if needed
- When adding new quants to existing test results without
- Improved osync ls Wildcard Handling - Better shell expansion handling on Linux/macOS
- Default behavior:
osync ls codematches models starting with "code" (prefix match, same ascode*) - Suffix match:
osync ls *q4_k_mfinds all models ending with "q4_k_m" (useful for finding by quantization) - Contains match:
osync ls *code*finds models containing "code" anywhere in the name - Shell expansion handling: detects when shell expanded unquoted wildcards and shows helpful warning
- Suggests using quotes to prevent expansion:
osync ls 'gemma*'
- Default behavior:
- Wildcard Tag Expansion for osync pull - Pull multiple models with tag patterns
- Supports wildcards in tags:
osync pull gemma3:1b-it-q*pulls all matching tags - Works with HuggingFace:
osync pull hf.co/unsloth/gemma-3-1b-it-GGUF:IQ2* - Works with remote servers:
osync pull -d http://server:11434 gemma3:1b-it-q* - Automatically resolves available tags from Ollama registry or HuggingFace API
- Shows list of matching tags before pulling
- Supports wildcards in tags:
- Judge Best Answer Tracking - QC judge now evaluates which response is qualitatively better
- Judge model returns
bestanswer: A (base better), B (quant better), or AB (tie) - Verbose output shows best answer for each judgment:
Score: 75% (27/50 54%) Best: AB - Handles edge cases: normalizes various formats (ResponseA, Response_A, Tie, identical, etc.)
- Results automatically re-judged if
--judgeis active andbestansweris missing
- Judge model returns
- QcView Judge Best Column - New column showing quant win statistics
- Format:
67% (B:10 A:5 =:3)showing quant won 67% of non-tie comparisons - B = quant better, A = base better, = = tie
- Best percentage excludes ties (only counts decisive wins/losses)
- Color-coded: green (>=60%), yellow (40-60%), red (<40%)
- Format:
- Enhanced JSON Output - Additional statistics in JSON export
- Per-question
BestAnswerfield (A/B/AB) - Per-quantization:
BestCount,WorstCount,TieCount,BestPercentage,WorstPercentage,TiePercentage - Per-category:
CategoryBestStatswith counts and percentages
- Per-question
- QcView Metrics-Only Mode - New
--metricsonlyargument to ignore judgment data- Shows only metrics-based scores (token similarity, logprobs divergence, perplexity, length consistency)
- Useful for comparing pure model output quality without judge influence
- Works with all output formats (table, json, md, html, pdf)
- Automatic Judge Context Length - Judge model context is now auto-calculated by default
- When
--judge-ctxsizeis 0 (new default), calculates: test_ctx × 2 + 2048 - Ensures judge has enough context for both base and quantized responses plus prompt
- Can still be manually overridden with explicit value
- When
- PDF Generation Progress Bar - Visual progress indicator when generating PDF reports
- Shows progress through Q&A pages for each quantization
- Useful for large test results files with many questions
- PDF Layout Improvements - Better page break handling in PDF reports
- Ranking tables use ShowEntire() to prevent splitting across pages
- Speed columns simplified to show only percentage (removed tok/s to prevent wrapping)
- Category scores section moves entirely to next page if it won't fit
- Rankings organized into paired rows (Final Score + Eval Speed, Perplexity + Prompt Speed, Best Answers)
- Added Prompt Speed ranking table with vs Base percentage column
- Manage TUI Batch Delete Fix - Fixed multi-selection delete not working
- Delete now properly handles multiple selected models (Ctrl+D with checkmarks)
- Confirmation dialog shows count and lists all models to be deleted
- Dialog title shows model count (e.g., "Confirm Delete (3 models)")
- Success message shows count of deleted models
- Partial success handling when some deletions fail
v1.2.5
- QC Resume Bug Fixes - Fixed critical issues with resuming from saved results files
- Fixed model name parsing when resuming: full model paths (e.g.,
hf.co/namespace/repo:tag) are now preserved correctly instead of being incorrectly derived from the-Margument - Fixed base model handling when resuming: the stored full model name is now used instead of just the tag portion
- Fixed verification loop to use stored model names from results file
- Fixed model name parsing when resuming: full model paths (e.g.,
- Cancellation Confirmation Prompt - Added y/n confirmation before cancelling QC tests
- First Ctrl+C now prompts "Cancel testing? (y/n)" instead of immediately cancelling
- Prevents accidental cancellation of long-running tests
- Second Ctrl+C still force exits immediately
v1.2.4
- New --rejudge Argument for QC Command - Re-run judgment without re-testing
- New
--rejudgeargument to re-run judgment process for existing test results - Unlike
--forcewhich re-runs both testing and judgment,--rejudgeonly re-runs judgment - Useful for re-evaluating results with a different judge model or updated prompts
- New
- Improved Judge Response Parsing - More robust handling of judge model responses
- Case-insensitive JSON property matching for Reason/reason/REASON fields
- Multiple regex patterns with increasing leniency for fallback parsing
- Truncated JSON repair to handle incomplete responses from models
- Increased num_predict from 200 to 800 to reduce truncated responses (later increased to 8192 in v1.2.9)
- Full raw JSON output displayed when reason parsing fails (for debugging)
- Improved Judge Scoring Accuracy - Fixed score interpretation issues
- Changed JSON schema score type from "integer" to "number" for better model compatibility
- Added explicit prompt instructions for 1-100 integer scoring (not 0.0-1.0 decimal)
- Score normalization to handle both 0.0-1.0 and 1-100 ranges from different models
- Fixed HuggingFace Model Verification in On-Demand Mode - Registry check now supports HuggingFace models
- On-demand mode (
--ondemand) now properly verifies HuggingFace models (hf.co/...) - Checks HuggingFace API to verify repository and GGUF file existence
- Supports various GGUF filename patterns for tag matching
- On-demand mode (
- Fixed Base Model Name Handling - Full model names now preserved for base tag
- Base model specified as full model name (e.g.,
-b qwen3-coder:30b-a3b-fp16or-b hf.co/namespace/repo:tag) is now used as-is - Previously, only the tag portion was extracted and combined with
-Mmodel name
- Base model specified as full model name (e.g.,
- Wildcard Tag Selection for QC Command - Dynamically select quantizations with patterns
- Support for wildcard patterns (
*) in-Qargument (e.g.,Q4*,IQ*,*) - Fetches available tags from HuggingFace API for
hf.co/...models - Scrapes available tags from Ollama website for Ollama registry models
- Case-insensitive pattern matching
- New
ModelTagResolverclass for reusable tag resolution across commands
- Support for wildcard patterns (
- Improved On-Demand Cleanup - Models pulled on-demand are now properly cleaned up on failure
- On-demand models are removed when testing fails or is cancelled
- Cleanup happens in exception handlers to ensure no orphaned models remain
- Models tracked at class level for reliable cleanup across failures
- Preload failures now also trigger cleanup of on-demand models
- Improved Model Preload - Better error handling and retry logic for model loading
- Added retry logic (3 attempts with exponential backoff) for transient failures
- Shows actual error message when preload fails (HTTP status, error details)
- Uses configurable timeout (
--timeout) for model loading - Handles timeout, connection errors, and server errors gracefully
- Fixed Model Name Case Sensitivity - Handle Ollama storing HuggingFace tags with different case
- After pulling, resolves actual model name stored by Ollama (case-insensitive lookup)
- Fixes issue where
Q4_0is stored asq4_0causing preload to fail
v1.2.3
- On-Demand Mode for QC Command - Pull models automatically and remove after testing
- New
--ondemandargument to enable on-demand model management - Models missing from the Ollama instance are automatically pulled from the registry
- Models that were already present are NOT removed (only on-demand pulled models)
- After testing and judgment complete, on-demand models are removed to free disk space
- State is persisted in results file for proper cleanup on resume
- Works with both local and remote Ollama servers
- Ideal for testing large models or many quantizations without consuming permanent storage
- New
- Context Length Support for QC Command - Configure context length (num_ctx) for testing and judgment
- Default test context length: 4096, default judge context length: 12288
- Suite-level
contextLengthproperty in built-in test suites (v1base, v1quick, v1code) - External JSON test suites support
contextLengthat suite, category, and question levels - Hierarchical override system: question > category > suite
- Console output displays context length at startup and when overridden
- New
--judge-ctxsizeargument to configure judge model context length
- Improved Console Output - Context length settings displayed during test execution
- Shows test and judge context lengths at the beginning of testing
- Displays override notifications when context length changes (e.g., "Context length changed to 8192 (from category Code)")
- Fixed Linux Terminal Display Issues - Resolved ANSI color rendering problems
- Fixed white box display issue in interactive REPL mode on Linux terminals
- Downgraded PrettyConsole (3.1.0 → 2.0.0) and Spectre.Console (0.54.0 → 0.49.1) for compatibility
- Improved Installer - Streamlined installation process
- Installer now only copies the main executable (no longer copies all directory files)
- Added mechanism for platform-specific optional dependencies
- Removed unnecessary libuv.dylib dependency for macOS (not needed in .NET 8+)
- Fixed Bash Completion on Linux
- Fixed tab completion for model names containing colons (e.g.,
osync ls qwen2:) - Fixed file tab completion for qcview command
- Fixed tab completion for model names containing colons (e.g.,
v1.2.2
- Improved Judgment Prompt Format - Better compatibility with more models
- Instructions now in both system prompt and user message for redundancy
- Clear text markers for RESPONSE A and RESPONSE B instead of JSON encoding
- Question included for context with clear delimiters
- Explicit rules to prevent models from judging quality/correctness instead of similarity
- Verbose Judgment Output - New
--verboseflag to show judgment details- Displays question ID, score (color-coded), and first 4 lines of reason
- Works with both serial and parallel judgment modes
- Helps debug and understand judge model scoring
- Fixed Parallel Verbose Output - Verbose output now displays during parallel judgment execution
- Previously showed all results after completion; now shows each result as it completes
- Fixed Serial Verbose Progress - Progress bar now displays alongside verbose output in serial mode
- Improved Cancellation Handling - Ctrl+C now immediately stops judgment without retrying
- Cancellation exceptions are no longer retried 5 times
- Judgment loop checks for cancellation before each question
- Missing Reason Retry - Judge API retries up to 5 times when response contains score but no reason
- Warning displayed if reason still missing after all retries
v1.2.1
- Bug Fix: Base Model Detection - Fixed issue where base model wasn't correctly identified when using full model names
- Base tag is now properly normalized when specified with full path (e.g.,
user/model:tag) - Existing results files with missing
IsBaseflag are automatically repaired on load - Judgment now correctly runs for quantizations that need it
- Base tag is now properly normalized when specified with full path (e.g.,
- Bug Fix: Output Filename Sanitization - Model names with
/or\are now converted to-in default output filename- Prevents file path issues when model name contains directory separators
- Improved Startup Output - Output file path is now displayed early in the execution
- Shows right after loading test suite, before judge model verification
- Cancellation Improvements - Better handling of Ctrl+C during API calls
- Cancellation token now passed to HTTP requests for immediate cancellation
- Wrapped cancellation exceptions are properly detected
v1.2.0
- Coding Test Suite - New
v1codetest suite for evaluating code generation quality- 50 challenging coding questions across 5 languages: Python, C++, C#, TypeScript, Rust
- Double token output limit (8192) for longer code responses
- Questions include instruction to limit response size
- Available as
-T v1codeor via externalv1code.jsonfile
- Configurable Token Output - Test suites now support custom
numPredictvalues- Each test suite can specify its own maximum token output
- External JSON test suites support
numPredictproperty (default: 4096) - Displayed in test suite info when non-default value is used
- Improved Model Existence Check - Pull command now uses Ollama registry API for faster, more reliable model validation
- Uses
registry.ollama.ai/v2/manifest endpoint instead of HTML scraping - Properly handles both library models and user models
- Faster response times and more accurate error messages
- Uses
- True Independent Parallel Judgment - Testing continues to next quantization while judgment runs in background
- Testing no longer waits for judgment to complete before moving to next quantization
- Background judgment tasks tracked and awaited at the end with progress display
- Progress bars show real-time status for both testing and judgment
- Improved Progress Display - Better visibility into parallel operations
- Dual progress bars during testing (Testing + Judging) in parallel mode
- Background judgment status shown after each quantization completes
- Final wait screen shows progress for all pending judgment tasks
- Configurable API Timeout - Added
--timeoutargument for testing and judgment API calls- Default increased from 300 to 600 seconds for longer code generation
- Configurable via
--timeout <seconds>argument - Applies to both test model and judge model API calls
- Resume Support - Gracefully handle interruptions and resume from where you left off
- Press Ctrl+C to save partial results and exit cleanly
- Re-run the same command to resume testing from the last saved question
- Partial quantization results are preserved in the JSON file
- Progress bar shows resumed position when continuing
- Missing judgments are automatically detected and re-run on resume
- UI Improvements
- Unified color scheme: lime for good scores (80%+) and performance above 100%
- Orange color for performance below 100%
v1.1.9
- Judge Model Scoring - Use a second LLM to evaluate quantization quality
--judge <model>- Specify local or remote judge model--mode serial|parallel- Serial (after each quant) or parallel (concurrent) execution- Judge evaluates similarity between base and quantized responses (score 1-100)
- Final score: 50% metrics + 50% judgment when enabled
- Supports local and remote judge models with auto-completion
- Results display shows separate Final, Metrics, and Judge scores
- True Parallel Judgment Mode - Testing and judgment now run concurrently at the question level
- In
--mode parallel, each question is immediately judged in the background as soon as testing completes - Test model continues generating answers while judge evaluates previous questions
- Significantly reduces total execution time when using a remote judge or fast judge model
- Parallel judgment also applies to existing quantizations that need re-judging
- In
- Improved Judge Response Handling
- Changed to
/api/chatendpoint with structured JSON output schema - Added
Reasonfield to capture judge's reasoning for each score - Robust parsing with fallbacks for different model response formats
- Treats score 0 as score 1 (minimum) for models that ignore instructions
- Changed to
- Enhanced Retry Logic
- Fixed retry display to show "Attempt 2/5", "Attempt 3/5" etc. after failures
- Added detailed error messages for judge API failures
- 5 retry attempts with exponential backoff
- Improved qcview Output - Enhanced file export functionality
- Fixed table file output with correct eval/prompt speed formatting
- Fixed JSON console output avoiding Spectre.Console markup parsing errors
- Added file size display on successful file creation
- Positional argument for results file (e.g.,
osync qcview results.json) - Displays judge model info when judgment scoring is available
- Bug Fixes
- Fixed judgment skip logic: existing quantizations without judgments or with different judge model are now properly re-judged
- Fixed parallel mode not executing judgment concurrently with testing
- Documentation Updates
- Updated color coding documentation to match actual thresholds
- Clarified qcview command usage and options
v1.1.8
- External Test Suite Support - Load custom test suites from JSON files using
-T path/to/suite.json- Bundled
v1base.jsonfile for reference and portability - Create custom test suites with your own categories and questions
- Full JSON schema support with categories, questions, and metadata
- Bundled
- Force Re-run Option - Added
--forceflag to re-run testing for quantizations already in results file - Improved Base Model Handling - Base tag defaults to
fp16and is automatically detected from existing results - Bug Fixes
- Fixed case-sensitive model name matching when retrieving disk size from
/api/tags - Fixed command argument positioning (PowerArgs uses 1-based indexing)
- Fixed
osync lsto accept model name as positional argument (e.g.,osync ls qwworks likeosync ls qw*)
- Fixed case-sensitive model name matching when retrieving disk size from
v1.1.7
- Quantization Comparison Testing - New
qcandqcviewcommands for comprehensive quality testing- 50-question test suite across 5 categories (Reasoning, Math, Finance, Technology, Science)
- 4-component weighted scoring: Logprobs Divergence (70%), Perplexity (20%), Token Similarity (5%), Length Consistency (5%)
- Performance metrics: eval/prompt tokens per second with percentage comparison to base model
- Incremental testing support - add new quantizations without re-testing existing ones
- JSON results file format with full question/answer history
- Color-coded table output in
qcview(90-100% green, 80-90% lime, 70-80% yellow, 50-70% orange, <50% red) - JSON export option for programmatic analysis
- Uses Ollama's logprobs API (requires Ollama v0.12.11+)
- Configurable test parameters: temperature, seed, top_p, top_k, repeat_penalty, frequency_penalty
- Works on both local and remote servers
- Bug Fixes
- Fixed PowerArgs duplicate alias errors by removing explicit shortcuts (auto-generated from property names)
- Fixed model detection issue:
/api/showendpoint doesn't includesizefield, now fetches from/api/tags - Fixed misleading "Results saved" message when no quantizations were successfully tested
- Added proper error handling with exit codes (returns 1 when no results)
- Set max output tokens to 4096 to prevent tests hanging on overly verbose answers
- Improved tag handling: accepts any tag format (with or without ":"), extracts quantization from API
v1.1.6
- Manage TUI Command - Full-screen interactive terminal user interface for model management
- Real-time model listing with dynamic column widths
- Multi-selection support for batch operations (copy, update, delete)
- Live filtering with instant search
- Six sort modes: Name+/-, Size+/-, Created+/- (with visual status indicator)
- Seven built-in themes (Default, Dark, Blue, Solarized, Gruvbox, Nord, Dracula)
- All operations accessible via keyboard shortcuts (Ctrl+C/M/R/S/D/U/P/L/K/X/O/T/Q)
- Enter key support in all dialogs for faster workflow
- Smart model selection persistence after operations
- Batch copy with incremental uploads
- Console output for long operations (update, pull) with auto-return to TUI
- Process Status Enhancements -
pscommand improvements- Shows VRAM usage percentage when model is partially loaded (e.g., "1.33 GB (63%)")
- Consistent tabular format in both CLI and manage TUI
- Displays model ID, size, context length, and expiration time
- Load/Unload Commands - VRAM memory management
loadcommand to preload models into memory with configurable keep-aliveunloadcommand to free VRAM immediately- Works on both local and remote servers
- Run/Chat Command - Interactive conversation with models
- Preloads model into memory before starting chat
- Shows loading status and model information
- Type
/byeor Ctrl+D to exit - Works on both local and remote servers
- Show Command - Display detailed model information
- Shows model metadata, configuration, and parameters
- Works on both local and remote servers
- Pull Command - Download models from Ollama registry
- Validates model existence on ollama.com before pulling
- Real-time progress display
- Supports library and user models
- Works on both local and remote servers
v1.1.0
- Remote-to-Remote Copy - Transfer models directly between remote servers without local storage
- Memory-Buffered Streaming - Efficient transfer using configurable memory buffer (default: 512MB)
- Simultaneous Download/Upload - Downloads from registry while uploading to destination
- Backpressure Control - Automatically throttles download when upload is slower
- Configurable Buffer Size - Use
-BufferSizeparameter (e.g.,256MB,1GB) - Enhanced Progress Display - Tqdm progress bar for remote-to-remote transfers matching local copy format
- Registry-Based Transfer - Pulls model blobs from Ollama registry for remote-to-remote operations
- Fixed HttpClient Issues - Resolved BaseAddress errors in list and copy commands
- Improved Error Messages - Clear documentation of remote-to-remote limitations
v1.0.9
- Added update command to update models to their latest versions
- Supports updating all models or specific patterns with wildcard matching
- Works on both local and remote Ollama servers
- Shows real-time progress and indicates whether each model was updated or already up to date
- Default behavior updates all models when no pattern specified
v1.0.8
- Added rename command with aliases
renandmvfor safe model renaming - Added local copy support - copy models locally, not just to remote servers
- Added sorting options for list command:
--size,--sizeasc,--time,--timeasc - Added destination existence checks for copy and rename operations to prevent overwriting
- Comprehensive README documentation with usage examples and patterns
v1.0.7
- Added list command with pattern matching support using
*wildcard - Remote listing support - list models on remote Ollama servers
- Multi-registry support - properly scans registry.ollama.ai, hf.co, hub, and custom registries
- Automatic
:latesttag fallback for copy and remove commands when tag not specified - Enhanced remove command with pattern matching support
- Fixed Ollama API migration for model creation (modelfile → files format)
- Fixed missing models in list output (models with
/in names)
v1.0.6
- Changed syntax to support multiple actions: now copying a model needs
copy(aliascp) - Fixed upload progress bar updates
- Handles automatically
latesttag if none specified
v1.0.5
- Added local models TabCompletion with interactive prompt if called without arguments
- Fixed 100 seconds timeout
- Added arguments exception handling
v1.0.4
- Check remote ollama version and display its version
- Fixed streaming output from create model
v1.0.3
- Added -bt switch to throttle the bandwidth in B, KB, MB, GB per second, eg. for 75MB/s use
-bt 75MB
v1.0.2
- Fixed build with single portable file for Linux/MacOS
v1.0.1
- Fixed bug with stdErr redirect
- Remove Linux and MacOS colored output
v1.0.0
- Initial release
Roadmap
See the open issues for a list of proposed features (and known issues).
- Top Feature Requests (Add your votes using the 👍 reaction)
- Top Bugs (Add your votes using the 👍 reaction)
- Newest Bugs
Support
License
This project is licensed under the MIT license.
See LICENSE for more information.
Similar repos
Languages
C#
98.1%
Gherkin
1.6%