README
Persona-ASR
Bilingual (Kazakh–English) target-speaker ASR for overlapping speech. Given a multi-talker mixture and a short enrollment utterance, the system transcribes only the enrolled speaker, and emits <no_target> instead of a transcript when that speaker is absent.
- Paper: R. Meiramov, T. Rakhimzhanova, A. Taibassarov, Z. Makhataeva, H. A. Varol. Persona-ASR: Bilingual Target-Speaker Speech Recognition for Kazakh–English Overlapping Speech. Machine Learning and Knowledge Extraction, 8(8), 246, 2026. doi:10.3390/make8080246
- Code: github.com/IS2AI/Persona_ASR
- Data: KazMix-3 · PersonaMix
Files
| File | Description |
|---|---|
asr_backbone.pt | ASR backbone: a frozen ECAPA-TDNN speaker embedding modulates a WavLM-Base-Plus encoder through FiLM, feeding language-specific CTC heads (English, Kazakh) and a frame-level VAD head. 118.67 M parameters. |
presence_gate.pt | Target-presence gate used in the paper: enrollment–mixture frame matching with attentive-statistics pooling, trained on the frozen backbone. 197,891 parameters. The validation-calibrated decision thresholds are stored in the checkpoint (τ_EN = 0.497, τ_KK = 0.609). |
presence_gate_dualpath_ablation.pt | Earlier two-path gate (matching + speaker-conditioned attention, 567,813 parameters), kept for the gate-design comparison in Section 5.4. It is not the published model. |
config.json | Backbone configuration. |
manifests/libri3mix/ | The English target-speaker manifests (train / validation / test) used in the paper, built on official Libri3Mix. Needed to reproduce the English results exactly. |
Correction (September 2026). Until this update,
presence_gate.ptin this repository was the two-path gate (nowpresence_gate_dualpath_ablation.pt), which does not correspond to the published results. If you downloaded the gate before September 2026, please download it again.
Results
Three-speaker test sets (Table 3 of the paper). Raw and gated WER are computed on target-present samples; detection metrics on target-present and target-absent samples.
| Test set | Raw WER (%) ↓ | Gated WER (%) ↓ | BAcc (%) ↑ | F1 (%) ↑ |
|---|---|---|---|---|
| English (Libri3Mix-100h) | 29.41 | 35.45 | 81.33 | 86.92 |
| Kazakh (KazMix3-100h) | 43.49 | 50.77 | 86.28 | 88.78 |
| Overall | 36.06 | 42.68 | 83.81 | 87.83 |
Training
The backbone is trained from scratch on target-present Libri3Mix (English) and KazMix-3 (Kazakh) mixtures with L = 0.7·CTC + 0.3·VAD for 100 epochs. The gate is trained on the frozen backbone with binary cross-entropy and a ranking loss for 40 epochs, on target-present and target-absent trials. Exact commands are in the GitHub repository.
Citation
@article{meiramov2026personaasr,
author = {Meiramov, Rakhat and Rakhimzhanova, Tomiris and Taibassarov, Adil and Makhataeva, Zhanat and Varol, Huseyin Atakan},
title = {Persona-ASR: Bilingual Target-Speaker Speech Recognition for Kazakh--English Overlapping Speech},
journal = {Machine Learning and Knowledge Extraction},
year = {2026},
volume = {8},
number = {8},
pages = {246},
doi = {10.3390/make8080246}
}
Funding
This research is funded by the Committee of Science of the Ministry of Science and Higher Education of the Republic of Kazakhstan (Grant No. BR24993001).
License
CC BY 4.0.
Similar repos
README
Persona-ASR
Bilingual (Kazakh–English) target-speaker ASR for overlapping speech. Given a multi-talker mixture and a short enrollment utterance, the system transcribes only the enrolled speaker, and emits <no_target> instead of a transcript when that speaker is absent.
- Paper: R. Meiramov, T. Rakhimzhanova, A. Taibassarov, Z. Makhataeva, H. A. Varol. Persona-ASR: Bilingual Target-Speaker Speech Recognition for Kazakh–English Overlapping Speech. Machine Learning and Knowledge Extraction, 8(8), 246, 2026. doi:10.3390/make8080246
- Code: github.com/IS2AI/Persona_ASR
- Data: KazMix-3 · PersonaMix
Files
| File | Description |
|---|---|
asr_backbone.pt | ASR backbone: a frozen ECAPA-TDNN speaker embedding modulates a WavLM-Base-Plus encoder through FiLM, feeding language-specific CTC heads (English, Kazakh) and a frame-level VAD head. 118.67 M parameters. |
presence_gate.pt | Target-presence gate used in the paper: enrollment–mixture frame matching with attentive-statistics pooling, trained on the frozen backbone. 197,891 parameters. The validation-calibrated decision thresholds are stored in the checkpoint (τ_EN = 0.497, τ_KK = 0.609). |
presence_gate_dualpath_ablation.pt | Earlier two-path gate (matching + speaker-conditioned attention, 567,813 parameters), kept for the gate-design comparison in Section 5.4. It is not the published model. |
config.json | Backbone configuration. |
manifests/libri3mix/ | The English target-speaker manifests (train / validation / test) used in the paper, built on official Libri3Mix. Needed to reproduce the English results exactly. |
Correction (September 2026). Until this update,
presence_gate.ptin this repository was the two-path gate (nowpresence_gate_dualpath_ablation.pt), which does not correspond to the published results. If you downloaded the gate before September 2026, please download it again.
Results
Three-speaker test sets (Table 3 of the paper). Raw and gated WER are computed on target-present samples; detection metrics on target-present and target-absent samples.
| Test set | Raw WER (%) ↓ | Gated WER (%) ↓ | BAcc (%) ↑ | F1 (%) ↑ |
|---|---|---|---|---|
| English (Libri3Mix-100h) | 29.41 | 35.45 | 81.33 | 86.92 |
| Kazakh (KazMix3-100h) | 43.49 | 50.77 | 86.28 | 88.78 |
| Overall | 36.06 | 42.68 | 83.81 | 87.83 |
Training
The backbone is trained from scratch on target-present Libri3Mix (English) and KazMix-3 (Kazakh) mixtures with L = 0.7·CTC + 0.3·VAD for 100 epochs. The gate is trained on the frozen backbone with binary cross-entropy and a ranking loss for 40 epochs, on target-present and target-absent trials. Exact commands are in the GitHub repository.
Citation
@article{meiramov2026personaasr,
author = {Meiramov, Rakhat and Rakhimzhanova, Tomiris and Taibassarov, Adil and Makhataeva, Zhanat and Varol, Huseyin Atakan},
title = {Persona-ASR: Bilingual Target-Speaker Speech Recognition for Kazakh--English Overlapping Speech},
journal = {Machine Learning and Knowledge Extraction},
year = {2026},
volume = {8},
number = {8},
pages = {246},
doi = {10.3390/make8080246}
}
Funding
This research is funded by the Committee of Science of the Ministry of Science and Higher Education of the Republic of Kazakhstan (Grant No. BR24993001).
License
CC BY 4.0.