Fibers of Asymmetric Similarity: A Framework for Clinical and Imaging Data

Asymmetric Tversky Similarity Fields

Prototype-based binary classification using asymmetric Tversky similarity fields. The method is general-purpose and applicable to any image or tabular classification task. It is evaluated here on two medical ultrasound benchmarks:

• CARDIUM — fetal cardiac ultrasound, CHD vs. control (dataset)
• ThyroidCineClip — thyroid cine-clip ultrasound, benign vs. malignant (dataset)

Two complementary pipelines are provided: one for image-based models and one for clinical tabular data.

Core Idea

Fig. 1 - Organ-wise Image Concepts and Patient-wise Concepts with Multiple-Instance Learning

Standard similarity metrics are symmetric — S(A, B) = S(B, A). Many classification problems are not. A borderline case may weakly resemble a positive-class prototype without that prototype resembling the borderline case. Asymmetric Tversky similarity captures this directional structure:

S(x → y) = |x ∩ y| / (|x ∩ y| + α|x \ y| + β|y \ x|)

α ≠ β breaks symmetry. During training, both S(x → prototype) and S(prototype → x) are computed and composed. The classifier is built from multiple composed similarity fields (Propositions 4 & 5): each field projects embeddings to a different subspace, learns its own prototype set, and the fields are combined via a learned convex weighting.

---

Image Pipeline

The image pipeline (train_similarity_field_v3_sigmoid.py) implements the multi-field Tversky classifier on top of frozen or fine-tuned pretrained encoders. Dataset-specific entry points wrap the same model code with the appropriate data loader and cross-validation strategy.

Architecture

Image → Pretrained Encoder → [num_fields × Projection Head] → Similarity Fields → Classification

• Encoders: medvit_original, medvit, visionclip, dinov2 (frozen or fine-tuned)
• Projection heads: mlp, grn, attention, se, bottleneck, cosine
• Output activation: sigmoid, softplus, relu, relu_norm — constrains features to [0, 1] for Tversky compatibility
• Prototypes: stored in logit space, decoded with sigmoid; separate positive (P1) and negative (P0) sets per field
• Patient-level evaluation: image scores aggregated per patient (mean or gated attention) before metrics are computed

Key Design Choices

Quick Start

# CARDIUM
python train_similarity_field_v3_sigmoid.py \
    --data_dir /path/to/CARDIUM \
    --encoders dinov2 \
    --feature_dim 128 \
    --num_prototypes 7 \
    --num_fields 3 \
    --alpha_values 0.8 \
    --beta_values 0.2 \
    --margin_values 0.3 \
    --projection_head mlp \
    --output_activation sigmoid \
    --epochs 100 \
    --use_cv \
    --patient_level_eval \
    --differential_lr \
    --class_weighted \
    --out results.json

# ThyroidCineClip
python train_similarity_field_v3_sigmoid_thyroid.py \
    --data_dir ./thyroidultrasoundcineclip \
    --encoders dinov2 \
    --feature_dim 128 \
    --num_prototypes 5 \
    --use_cv \
    --num_cv_folds 5 \
    --out thyroid_results.json

Parallel Grid Search

bash run_experiments_v3_sigmoid.sh --gpus 0,1,2,3 --jobs-per-gpu 2

Sweeps encoders × feature dims × prototype counts × α/β × margin across GPUs and writes per-experiment JSON results.

Key Arguments

---

Tabular Pipeline

The tabular pipeline treats clinical records as unordered sets of feature–value pairs, making the model invariant to measurement ordering and robust to variable-length records. The same architecture is evaluated on both benchmarks.

Architecture

Clinical record (set of measurements)
    → Type-specific embedders (numeric / categorical / coded / temporal)
    → Permutation-invariant aggregation (Deep Sets: ρ ∘ Σφ)
    → [optional] Hypergraph layer (patient–feature–cell relationships)
    → Asymmetric Tversky similarity classifier

Type-specific modules:

A clinical schema object encodes domain knowledge: valid ranges, feature types, monotonicity directions, and ontology hierarchy depths — but this can be replaced with a schema appropriate for any domain.

Quick Start

# CARDIUM
python benchmark_patient_level_v2.py \
    --data_dir /path/to/CARDIUM \
    --epochs 50 \
    --batch_size 32

# ThyroidCineClip
python benchmark_thyroid_v2.py \
    --data_dir ./thyroidultrasoundcineclip \
    --epochs 50 \
    --batch_size 32

---

Evaluation Datasets

The datasets below are used to evaluate the method; they are not requirements for using the code.

CARDIUM — fetal cardiac ultrasound + clinical records:

cardium_data/
├── cardium_images.tar.gz
├── delfos_clinical_data_woe_wnm_standarized_f_normalized.json
└── delfos_clinical_data_wnm_translated_final_cleaned.json

Download: https://github.com/BCV-Uniandes/Cardium

ThyroidCineClip — thyroid ultrasound cine-clips + metadata:

thyroidultrasoundcineclip/
├── dataset.hdf5
└── metadata.csv

Download: https://stanfordaimi.azurewebsites.net/datasets/thyroid-disease

---

Metrics

All evaluation is patient-level: image-level scores are aggregated per patient before computing metrics.

Results are reported as mean ± std across cross-validation folds.