cFreD -- Conditional Frechet Distance

This is the official implementation of Evaluating Text-to-Image Synthesis with a Conditional Fréchet Distance (WACV 2026).

How cFreD Works

Standard FID measures how well generated images match a reference distribution, but it ignores the text prompts that produced them. Two models could achieve the same FID even if one follows prompts faithfully and the other ignores them entirely. cFreD fixes this by conditioning on the prompt embeddings, measuring distributional similarity given the text.

Given:

• x — text (prompt) embeddings
• y — reference image/video embeddings
• $\hat{y}$ — generated image/video embeddings

cFreD models $(x, y)$ and $(x, \hat{y})$ as joint Gaussians and computes the Frechet distance between their conditional distributions $y|x$ and $\hat{y}|x$:

$$\text{cFreD} = | \mu_y - \mu_{\hat{y}} |^2 + \text{tr}\left((\Sigma_{yx} - \Sigma_{\hat{y}x}), \Sigma_{xx}^{-1}, (\Sigma_{xy} - \Sigma_{x\hat{y}})\right) + \text{tr}\left(\Sigma_{yy|x} + \Sigma_{\hat{y}\hat{y}|x} - 2\left(\Sigma_{yy|x}^{1/2}\ \Sigma_{\hat{y}\hat{y}|x}\Sigma_{yy|x}^{1/2}\right)^{1/2}\right) $$

where:

• $\mu_y$, $\mu_{\hat{y}}$ are the means of $y$ and $\hat{y}$
• $\Sigma_{xx}$ is the covariance of $x$; $\Sigma_{yx}$, $\Sigma_{\hat{y}x}$ are the cross-covariances
• $\Sigma_{yy|x}$ is the conditional covariance of $y$ given $x$

Lower is better. A score of 0 means the conditional distributions are identical.

Embedding Models

cFreD uses the following models by default:

• Image: timm/vit_giant_patch14_dinov2.lvd142m (DINOv2 ViT-Giant) — a self-supervised vision transformer that produces rich visual features without relying on text-image contrastive training.
• Text: convnext_base_w.laion2b_s13b_b82k_augreg (OpenCLIP ConvNeXt-Base text encoder) — the text tower of an OpenCLIP model trained on LAION-2B.

Installation

pip install -r requirements.txt

For video mode, you also need OpenCV:

pip install opencv-python

Data Setup

Image Mode

Organize your data as two flat directories of images plus a prompt file:

my_dataset/
├── ref/                 # --ref-dir
│   ├── 00001.png
│   ├── 00002.png
│   └── 00003.jpg
├── gen/                 # --gen-dir
│   ├── 00001.png        # matched to ref/00001.png by stem "00001"
│   ├── 00002.jpg        # extensions CAN differ
│   └── 00003.png
└── prompts.json         # --prompts

Matching rules:

• Images are matched between --ref-dir and --gen-dir by filename stem (name without extension). ref/cat.png pairs with gen/cat.jpg.
• Only stems present in both directories are used; extras are silently skipped.
• Matched pairs are sorted alphabetically by stem.
• Extensions can differ between directories. Supported formats: bmp, jpg, jpeg, pgm, png, ppm, tif, tiff, webp.
• Both directories must be flat (no subdirectories are scanned).

Video Mode

my_video_dataset/
├── ref_videos/          # --ref-dir
│   ├── 000.mp4
│   ├── 001.mp4
│   └── 002.mp4
├── gen_videos/          # --gen-dir
│   ├── 000.mp4
│   ├── 001.mp4
│   └── 002.mp4
└── prompts.json         # --prompts

Matching rules:

• Only .mp4 files are detected.
• Videos are matched positionally (sorted by filename), not by stem matching.
• All three lists (ref videos, gen videos, prompts) are truncated to the shortest.
• Each video must have at least --num-frames frames (default 16) or it is skipped during embedding.

Prompt Formats

The --prompts flag accepts JSON, CSV, or plain-text files.

JSON (list of strings)

The simplest format:

[
  "a photo of a cat sitting on a couch",
  "a red sports car on a highway",
  "a painting of a sunset over the ocean"
]

JSON (list of dicts)

[
  {"prompt": "a photo of a cat sitting on a couch"},
  {"prompt": "a red sports car on a highway"},
  {"prompt": "a painting of a sunset over the ocean"}
]

The "prompt" or "Prompt" key is required; any other keys are ignored.

CSV

Must have a prompt or Prompt column header:

prompt
a photo of a cat sitting on a couch
a red sports car on a highway
a painting of a sunset over the ocean

TXT

One prompt per line:

a photo of a cat sitting on a couch
a red sports car on a highway
a painting of a sunset over the ocean

Ordering

The i-th prompt in the file must correspond to the i-th image pair (sorted alphabetically by stem).

For example, if matched stems are ["00001", "00002", "00003"], then:

If there are more prompts than matched image pairs (or vice versa), the CLI truncates to the shorter list with a warning.

Quick Start (CLI)

Image mode

python -m cfred.compute_cfred \
    --ref-dir ./my_dataset/ref \
    --gen-dir ./my_dataset/gen \
    --prompts ./my_dataset/prompts.json \
    --batch-size 32 \
    --device cuda:0 \
    --json-out results.json

Video mode

python -m cfred.compute_cfred \
    --ref-dir /path/to/ref_videos \
    --gen-dir /path/to/gen_videos \
    --prompts prompts.json \
    --mode video \
    --num-frames 16

Quick Start (Python API)

from cfred.image_models import get_image_embedder
from cfred.text_models import get_text_embedder
from cfred.embeddings import load_image_embeddings, load_text_embeddings, match_files, load_prompts
from cfred import cfred

img_model = get_image_embedder("timm/vit_giant_patch14_dinov2.lvd142m", device="cuda")
text_model = get_text_embedder("convnext_base_w.laion2b_s13b_b82k_augreg", device="cuda")

ref_paths, gen_paths = match_files("path/to/ref", "path/to/gen")
prompts = load_prompts("prompts.json")

y_true = load_image_embeddings(ref_paths, img_model)
y_predict = load_image_embeddings(gen_paths, img_model)
x_true = load_text_embeddings(prompts, text_model)

score = cfred(y_true, y_predict, x_true)

Using Different Embedding Models

You can experiment with other supported models via --img-model and --text-model:

python -m cfred.compute_cfred \
    --ref-dir ./my_dataset/ref \
    --gen-dir ./my_dataset/gen \
    --prompts ./my_dataset/prompts.json \
    --img-model timm/vit_base_patch14_reg4_dinov2.lvd142m \
    --text-model FacebookAI/roberta-base

To list all available models:

python -m cfred.compute_cfred --list-models

Citation

@inproceedings{koo2026evaluating,
  title={Evaluating Text-to-Image and Text-to-Video Synthesis with a Conditional Fr{\'e}chet Distance},
  author={Koo, Jaywon and Hernandez, Jefferson and Haji-Ali, Moayed and Yang, Ziyan and Ordonez, Vicente},
  booktitle={Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision},
  pages={2052--2062},
  year={2026}
}