marian-uk-verbalizer-spans, CTranslate2 INT8
0
2 commits
2 repos using this model
3 linked in READMEs
updated Oct 14, 2026
A 52.7M-parameter Marian model (E60) that turns the written parts of a Ukrainian sentence -- numbers, dates, money,
units, Roman numerals, abbreviations, Latin words, symbols, footnote marks -- into the words a narrator would say,
one span at a time. It sees the whole sentence with one span marked [[ ]] and writes only that span's spoken
form; everything outside the spans is copied from the input, so ordinary prose is never rewritten.
На 2 поверсі є 3 кімнати та 12 дверей.
spans: [[2 поверсі]] [[3 кімнати]] [[12 дверей]]
model: другому поверсі | три кімнати | дванадцять дверей
-> На другому поверсі є три кімнати та дванадцять дверей.
Вічний Жид[93], і в мене також є свій ґедзь. [[[93]]] -> "" (footnote marks are not read)
spans.py (next to the model) finds the spans from the shape of the characters only and assembles the result; it is
the reference implementation used by the accentuk package. Training and
evaluation code: vacuum-in/marian-uk-verbalizer.
import ctranslate2
from transformers import MarianTokenizer
from spans import assemble, detect_spans, model_input
tok = MarianTokenizer.from_pretrained(".")
tr = ctranslate2.Translator(".", compute_type="int8")
def verbalize(sentence):
spans = detect_spans(sentence)
inputs = [tok.convert_ids_to_tokens(tok(model_input(sentence, sp), truncation=True, max_length=192)["input_ids"])
for sp in spans]
res = tr.translate_batch(inputs, beam_size=1, max_decoding_length=96,
min_decoding_length=0, # a span may be read as nothing
suppress_sequences=[["<s>"]]) if inputs else []
outs = [tok.decode(tok.convert_tokens_to_ids(r.hypotheses[0]), skip_special_tokens=True) for r in res]
return assemble(sentence, spans, outs)
min_decoding_length=0 is required: CTranslate2's default of 1 forbids the empty reading of a footnote mark, and the
model then invents a word for it. suppress_sequences=[["<s>"]]: this export starts decoding from <s>, which must
never be generated.
Spoken match (case, punctuation and hyphens are not heard), measured 2026-10-14:
| test | this model (CT2 INT8, CPU) | E54, whole sentence | Gemma-3-270M verbalizer, whole sentence |
|---|---|---|---|
held-out splits of skypro1111/uk-text-normalization, 3,619 rows | 88.86% | 89.53% | ~90.2% |
| 1,102 audiobook sentences with digits, Latin, abbreviations or symbols (references by Qwen3.8-27B) | 70.05% | 62.61% | 58.89% |
| audiobook sentences with a word outside the spans altered | 0.45% | 4.99% | 8.08% |
10.2 ms per sentence on a laptop CPU (batched spans), 54 MB.
E54 (the whole-sentence model of this project) warm-started on 1.1M span pairs cut from E54's sentence pairs by aligning written and spoken words, plus footnote marks read as nothing, ruler names before Roman numerals in every case and animate counts upsampled; two epochs at 5e-5.
marian-uk-verbalizer-spans, CTranslate2 INT8
0
2 commits
2 repos using this model
3 linked in READMEs
updated Oct 14, 2026
A 52.7M-parameter Marian model (E60) that turns the written parts of a Ukrainian sentence -- numbers, dates, money,
units, Roman numerals, abbreviations, Latin words, symbols, footnote marks -- into the words a narrator would say,
one span at a time. It sees the whole sentence with one span marked [[ ]] and writes only that span's spoken
form; everything outside the spans is copied from the input, so ordinary prose is never rewritten.
На 2 поверсі є 3 кімнати та 12 дверей.
spans: [[2 поверсі]] [[3 кімнати]] [[12 дверей]]
model: другому поверсі | три кімнати | дванадцять дверей
-> На другому поверсі є три кімнати та дванадцять дверей.
Вічний Жид[93], і в мене також є свій ґедзь. [[[93]]] -> "" (footnote marks are not read)
spans.py (next to the model) finds the spans from the shape of the characters only and assembles the result; it is
the reference implementation used by the accentuk package. Training and
evaluation code: vacuum-in/marian-uk-verbalizer.
import ctranslate2
from transformers import MarianTokenizer
from spans import assemble, detect_spans, model_input
tok = MarianTokenizer.from_pretrained(".")
tr = ctranslate2.Translator(".", compute_type="int8")
def verbalize(sentence):
spans = detect_spans(sentence)
inputs = [tok.convert_ids_to_tokens(tok(model_input(sentence, sp), truncation=True, max_length=192)["input_ids"])
for sp in spans]
res = tr.translate_batch(inputs, beam_size=1, max_decoding_length=96,
min_decoding_length=0, # a span may be read as nothing
suppress_sequences=[["<s>"]]) if inputs else []
outs = [tok.decode(tok.convert_tokens_to_ids(r.hypotheses[0]), skip_special_tokens=True) for r in res]
return assemble(sentence, spans, outs)
min_decoding_length=0 is required: CTranslate2's default of 1 forbids the empty reading of a footnote mark, and the
model then invents a word for it. suppress_sequences=[["<s>"]]: this export starts decoding from <s>, which must
never be generated.
Spoken match (case, punctuation and hyphens are not heard), measured 2026-10-14:
| test | this model (CT2 INT8, CPU) | E54, whole sentence | Gemma-3-270M verbalizer, whole sentence |
|---|---|---|---|
held-out splits of skypro1111/uk-text-normalization, 3,619 rows | 88.86% | 89.53% | ~90.2% |
| 1,102 audiobook sentences with digits, Latin, abbreviations or symbols (references by Qwen3.8-27B) | 70.05% | 62.61% | 58.89% |
| audiobook sentences with a word outside the spans altered | 0.45% | 4.99% | 8.08% |
10.2 ms per sentence on a laptop CPU (batched spans), 54 MB.
E54 (the whole-sentence model of this project) warm-started on 1.1M span pairs cut from E54's sentence pairs by aligning written and spoken words, plus footnote marks read as nothing, ruler names before Roman numerals in every case and animate counts upsampled; two epochs at 5e-5.