Ukrainian stress combiner (combiner-v1)

A small learned scorer over the pipeline's tiers. For each candidate reading of an ambiguous word, it sees:

• what every tier said about it;
• which reading the fixed tier order chose.

It overrules the tier order only when it prefers another reading by a margin.

Input and output

32 features per candidate. wiki-stress/ml/src/ukstress_ml/combiner.py defines them; model.pt carries the list, and the list is part of the model:

• Model: one hidden layer of 32 tanh units, softmax across a token's candidates.
• Output: a reading and its probability.
• Gate τ = 0.2: it departs from the tier order only when its preferred reading's probability beats the tier order's reading by τ.
• Unseen tiers: tiers that were added after training (the agreement repair) are always kept.

from ukstress_ml.combiner import Combiner, Assets
combiner = Combiner("uk-stress-combiner/model.pt")
assets = Assets.load("uk-stress-combiner")   # readings.jsonl, audio_prior.json, classifier_profile.json

In the API it is off by default. A request enables it with "combiner": true.

Training

• Data: labelled tokens from two sources.
  • Common Voice uk, with audio gold at ranker confidence ≥ 0.99, split 60/20/20 by sentence.
  • The dev half of the lang-uk benchmark, human gold, weighted ×4.
• Features: computed by running every tier independently over those tokens.
• Model selection: the hidden size and weight decay were chosen on dev (cv + lang-uk). τ was then chosen as the largest dev gain on Common Voice that costs lang-uk no more than 0.3 points.
• Hardware: CPU, about 1 min.
• Reproducibility: retrained from the same features, it is byte-identical to the released model.pt.

Evaluation

Held-out test splits, the tier order against the combiner:

Limitations

• Tied to the other models. It was trained on the outputs of these exact tier models. A different cross-encoder or classifier shifts its features and needs a retrained combiner.
• Modern-text gold. It helps on audio-gold tests. On the project's small modern-text gold it scored 97.27% against the tier order's 98.07%, so it ships switched off.
• Features at request time. It needs every tier's opinion, which the service computes once per request and caches.