SignSparK-BT

Back-translation models for evaluating sign language production. Each model takes SMPL+MANO pose features and produces spoken language text, so generated sign sequences can be scored by translating them back and comparing against the source sentence.

Here, we release the backtranslation models for SignSparK, which read from the same pose data as in LionelLow/SignSparK_data.

Code: JianHe0628/SignSparK_BT

Usage

pip install git+https://github.com/JianHe0628/SignSparK_BT
python tools/download_models.py --datasets PHOENIX-2014T CSL-Daily --dest ./checkpoints

Score a SignSparK .npy sample dump, reporting ground truth, generation and the drop between them:

signspark-bt score checkpoints/PHOENIX-2014T \
    --body-npy  .../body/<run>/seed102_clampstep0_PHOENIX-2014T.npy \
    --hand-npy  .../hand/<run>/seed102_clampstep0_PHOENIX-2014T.npy

Or from Python:

import torch
from signspark_bt import make_back_translation_model, back_translate

model = make_back_translation_model("checkpoints/PHOENIX-2014T")

# B x T x 240: body (60) + left hand (90) + right hand (90)
sentences = back_translate(model=model, poses=torch.rand(8, 64, 240))

Results

Ground-truth poses on the full test split. These are the upper-bound values: the best any generated sequence should theoretically score through our BT models.

CSL-Daily is scored per character, so its chrF is not comparable with the other datasets.

Models

Each directory holds config.yaml, best.ckpt, gls.vocab and txt.vocab.

Input features

240 dimensions per frame, drawn from the SignSparK LMDBs:

SignSparK .npy sample dumps store the left hand conjugated into right-hand convention; the loader undoes this. The face stream is not used.

Limitations

Note that backtranslation scores are heavily dependant on which backtranslation model and data representation you use.

This means that BLEU scores across papers are often hard to compare against, unless they have used the same model checkpoints.

This can be slightly remedied by reporting the relative percentage drops, as described in our paper. However, even so, backtranslation can still often be limited by the following:

• Since it is a language model, it may possibly produce a fluent sentence from bad poses.
• Sign Language Translation is still an open research problem; thus, the SLT model can be an unreliable proxy for evaluating production.

We recommend any use of backtranslation scores to be reported with its relative drop together with human evaluation and pose metrics.

Additionally, note that our released checkpoints here have all been trained on the re-optimized poses in LionelLow/SignSparK_data. Older checkpoints trained on the raw data are deprecated.

Citation

@inproceedings{low2026signspark,
  title={SignSparK: Efficient Multilingual Sign Language Production via Sparse Keyframe Learning},
  author={Low, Jianhe and Symeonidis-Herzig, Alexandre and Ivashechkin, Maksym and Sincan, Ozge Mercanoglu and Bowden, Richard},
  booktitle={European Conference on Computer Vision},
  pages={648--670},
  year={2026},
  organization={Springer}
}