qc - Video Quality Control

Fast, statistically honest video quality analysis: a Go library and a CLI that take a video and give you technical metrics, a VMAF score and a per-title adaptive streaming ladder (H.264, HEVC, AV1) as fast as the reliability you ask for allows.

• Technical analysis in seconds: bitrate, peaks and GOP structure without decoding. Then one decode fanned out to SI/TI (ITU-T P.910), shot detection, black and frozen segments, letterbox/pillarbox, luma levels and camera motion (each shot static, pan, tilt, zoom, tracking or handheld, with a shake measure).
• Audio QC at no extra wall time: every audio track measured while the video decodes: loudness per ITU-R BS.1770-5 (integrated, range, true peak, EBU Tech 3341/3342 conformant, within 0.01 LU of ffmpeg's ebur128) against EBU R 128, ATSC A/85 or a streaming target; silence, muted channels, clipping, DC offset and phase problems (audio).
• VMAF with a confidence interval: short clips sampled across shots until the 95% interval is narrower than your target. The intervals really cover the truth 95% of the time, measured by replaying thousands of runs. --exact scores every frame, bit-exact with Netflix's vmaf tool, at 8 and 10 bits. Or pick a fixed budget (--sample 2%, --sample 1/scene).
• Beyond VMAF, on the same frames: XPSNR (a pure-Go port matching ffmpeg's filter), CAMBI banding, PSNR, PSNR-HVS, SSIM, MS-SSIM, CIEDE2000 or the whole AV2 CTC set, each with its own confidence interval, and VMAF per viewing device (phone, TV, 4K).
• Per-title ladders, verified: probe encodes of a representative digest, rate-quality curves per resolution, their upper envelope, rungs one just-noticeable difference apart, then a real encode of every rung. It lands on the exhaustive optimum (−0.04 VMAF, −0.7% bitrate on average) in a fraction of the time. Impose the shape (--rungs 1080,720,540), probe adaptively where the rungs are uncertain, add per-shot rungs, or let AV1 synthesise film grain; verified rungs are checked for banding and for VMAF/XPSNR disagreements.
• HDR aware: HDR10/HLG checked, MaxCLL/MaxFALL measured, wPSNR and ΔE ITP next to VMAF, 10-bit ladders carrying the HDR10 metadata (HDR).
• See what was measured: --overlay annotated.mp4 burns the analysis into a copy of the video, frame by frame: timecode, bitrate, shots, camera motion, SI/TI, levels, VMAF of each scored frame and a timeline (annotated videos).
• A terminal UI you'll enjoy: a live dashboard with progress, ETA and panels that show VMAF converging and probes landing on a braille chart. There is also an interactive wizard, plus JSON and self-contained HTML reports.

Install

Docker (linux/amd64, linux/arm64): qc, libvmaf 3.2.1 with its models and ffmpeg with x264, x265 and SVT-AV1, nothing else to install. Mount your videos on /data:

docker run --rm -v "$PWD:/data" ghcr.io/eko/qc vmaf reference.mov encode.mp4
docker run --rm -v "$PWD:/data" ghcr.io/eko/qc ladder source.mov -c av1 --html ladder.html

Homebrew (macOS, Linux):

brew install eko/tap/qc

Prebuilt binaries (Linux amd64/arm64, static; macOS arm64): on every release, with libvmaf and the VMAF models built in; only ffmpeg is needed (install).

From source: Go (see go.mod) with cgo, ffmpeg and ffprobe with libx264, libx265 and libsvtav1, libvmaf ≥ 3.2.1 with its models and pkg-config:

brew install ffmpeg libvmaf pkgconf   # macOS; Debian/Ubuntu: see docs/install.md
go install github.com/eko/qc/cmd/qc@latest

qc version --check checks the installation. Linux prerequisites, the image contents and troubleshooting: docs/install.md.

Quick start

qc                                                        # interactive wizard
qc run source.mov --codecs h264,av1 --html report.html    # analysis + ladders
qc run encode.mp4 -r source.mov                           # + VMAF against the source

qc analyze video.mp4 [--fast]                             # technical analysis (--fast: no decoding)
qc analyze video.mp4 --loudness-target atsc               # audio checked against ATSC A/85 (default: EBU R 128)
qc vmaf reference.mov distorted.mp4 [--exact]             # VMAF ± 95% CI, or every frame
qc vmaf reference.mov distorted.mp4 --sample 5%           # fixed budget (or 2/scene), one pass, CI reported
qc vmaf reference.mov distorted.mp4 --exact --overlay annotated.mp4   # + a copy with per-frame VMAF burnt in
qc ladder source.mov -c av1 --encode-bit-depth 10         # per-title Main10 AV1 ladder
qc ladder source.mov --rungs 1080,720,540,360 --top-vmaf 93   # impose the rungs, bitrates computed
qc ladder source.mov -c av1 --encode-ladder renditions/    # + the renditions, checked on the whole title

Every command accepts -o report.json, --html report.html and -f json. See the CLI reference.

Performance

Apple M2 Max, real 1080p25 H.264 sources:

How these numbers were obtained, and how to reproduce them on your own content: docs/validation.md.

Documentation

• Install
• How it works
• Architecture
• Technical analysis
• VMAF engine
• HDR: detection, light levels, HDR metrics, HDR ladders
• Audio: loudness (BS.1770, EBU R 128, ATSC A/85) and defects
• Ladder engine
• Validation
• CLI
• NVIDIA GPUs: NVDEC decoding, NVENC ladders and CUDA VMAF with --gpu
• Annotated videos: the analysis burnt into a copy of the video with --overlay
• Innovation landscape and roadmap: beyond VMAF

Library

dec := decode.NewFFmpeg("ffmpeg", 0) // decode.WithHWAccel(decode.HWAccelAuto): VideoToolbox segments on macOS; HWAccelCUDA: NVDEC
analyzer := analysis.New(logger,
	probe.NewFFprobe("ffprobe"),
	bitstream.NewFFprobeReader("ffprobe"),
	dec,
	quality.NewMeter(dec, libvmaf.NewEngine()),
)

report, err := analyzer.Analyze(ctx, "video.mp4", analysis.Options{})
cmp, err := analyzer.Compare(ctx, "reference.mov", "encode.mp4", analysis.CompareOptions{})
ffmpeg := encode.NewFFmpeg("ffmpeg") // encodes, digests, grain measurements, renditions
engine := ladder.NewEngine(analyzer, ffmpeg, ffmpeg,
	ladder.WithGrainLab(ffmpeg), ladder.WithRenditionEncoder(ffmpeg))
res, err := engine.Build(ctx, "source.mov", ladder.Options{Codec: "av1"})

Every options struct has a useful zero value, except that ladder.Options needs its Codec. To go further:

cmp, err := analyzer.Compare(ctx, "reference.mov", "encode.mp4", analysis.CompareOptions{
	Quality: quality.Options{
		Precision: 0.25,                                         // or Exact: true, or Sample below
		Metrics:   []string{quality.MetricXPSNR, quality.MetricCAMBI},
		Devices:   []string{vmaf.DevicePhone, vmaf.Device4K},
	},
})
xpsnrY, _ := cmp.VMAF.Metric(quality.SeriesXPSNRY)  // mean and 95% CI, like VMAF
phone, _ := cmp.VMAF.Device(vmaf.DevicePhone)
banding := cmp.VMAF.Banding                         // segments where CAMBI > 5

// A fixed budget instead of a precision; the analysed encode gives its shot
// cuts to per-scene budgets and is not inspected again.
budget, _ := quality.ParseSample("2/scene")        // or "5%"
cmp, err = analyzer.Compare(ctx, "reference.mov", "encode.mp4", analysis.CompareOptions{
	Quality:   quality.Options{Sample: budget},
	Distorted: report,
})

// An imposed AV1 ladder, probed adaptively, with checked rungs.
res, err = engine.Build(ctx, "source.mov", ladder.Options{
	Codec:       "av1",
	Constraints: ladder.Constraints{Resolutions: []int{1080, 720, 540, 360}, TopVMAF: 93},
	Probing:     ladder.ProbingAdaptive,
	FilmGrain:   ladder.FilmGrainAuto,
	Metrics:     []string{quality.MetricXPSNR, quality.MetricCAMBI},
})
banded := ladder.BandedRungs(res.Rungs)        // rungs capped by banding
conflicts := ladder.RankConflicts(res.Rungs)   // VMAF and XPSNR disagree

// Per-shot rungs: one CRF per shot at an equal rate-quality slope.
res, err = engine.Build(ctx, "source.mov", ladder.Options{Codec: "hevc", PerShot: true})
for shot, cell := range res.ShotLadder(0) {    // the top rung, shot by shot
	fmt.Println(res.ShotInterval(shot).Start, cell.CRF, cell.PredictedBitrate)
}
if top := res.Rungs[0].PerShot; top != nil {
	fmt.Println(top.PooledBitrate(res.Shots))  // its bitrate over the whole title
}

// The renditions: every rung (and per-shot version) encoded on the whole
// title, each checked against the source.
renditions, err := engine.Encode(ctx, "source.mov", res, ladder.RenditionOptions{
	Dir:   "renditions",
	Check: &quality.Options{Precision: 0.5},
})
for _, rd := range renditions {
	vmafPredicted, bitratePredicted := res.Prediction(rd)
	fmt.Println(rd.Path, rd.Bitrate, bitratePredicted, rd.Checked.VMAFLabel(), vmafPredicted)
}

On an NVIDIA GPU, check first: NVDEC falls back to the CPU by itself, but an NVENC ladder would fail at its first encode. CUDA VMAF needs a binary built with -tags cuda against a CUDA libvmaf (libvmaf.CUDABuilt), and a model with CUDA features (VMAF v1, the default, has none):

err = nvidia.Check(ctx, "ffmpeg", nvidia.Requirements{
	HWAccel: decode.HWAccelCUDA,   // the decoder's decode.WithHWAccel
	Codecs:  []string{"hevc"},     // ladders on NVENC
})
res, err = engine.Build(ctx, "source.mov", ladder.Options{
	Codec:   "hevc",
	Encoder: encode.HardwareNVENC, // no per-shot rungs nor film grain synthesis
	Backend: vmaf.BackendAuto,     // CUDA VMAF when the model and the build allow it
})
fmt.Println(cmp.VMAF.GPUSummary()) // e.g. "NVDEC decoding (cuda) · VMAF features on CUDA"

quality/xpsnr also works on its own, on decoded frames, and matches ffmpeg's xpsnr filter. Runnable examples are on pkg.go.dev for analysis, quality, quality/xpsnr, quality/hdr, ladder, pipeline, nvidia, overlay and vmaf/libvmaf. To run everything with progress hooks, use pipeline.Runner as described in docs/architecture.md.

Contributing

Contributions are welcome. See CONTRIBUTING.md: make check runs what CI runs, and changes to the sampler or the ladder engine come with their validation numbers.

License

MIT