dynamic-metric-evaluation

Usage

To evaluate translation systems, you can use the evaluate.py script:

python dmm/evaluate.py -l en-cs -m chrf -d wmt24

References

Evaluation (human score) data from https://github.com/google-research/mt-metrics-eval

@misc{wmt24pp,
      title={{WMT24++: Expanding the Language Coverage of WMT24 to 55 Languages & Dialects}}, 
      author={Daniel Deutsch and Eleftheria Briakou and Isaac Caswell and Mara Finkelstein and Rebecca Galor and Juraj Juraska and Geza Kovacs and Alison Lui and Ricardo Rei and Jason Riesa and Shruti Rijhwani and Parker Riley and Elizabeth Salesky and Firas Trabelsi and Stephanie Winkler and Biao Zhang and Markus Freitag},
      year={2025},
      eprint={2502.12404},
      archivePrefix={arXiv},
      primaryClass={cs.CL},
      url={https://arxiv.org/abs/2502.12404}, 
}