Installation

To install the necessary packages and their adequate Python versions, we recommend the use of Anaconda to create independent environments.

Moshi

This section describes how to install the moshi package extracted from the GitHub repository of the Kyutai team (see here). To install the package, you can run the following commands:

conda create -n moshi
conda activate moshi
pip install -e moshi/

Metrics

We recommend creating another virtual environment to compute the different metrics.

conda create -n metrics
pip install -r requirements.txt

You can compute the following metrics with their related Python scripts:

Previous Installations

In this section, we show how former models used for metrics computation were installed.

Mean Opinion Score

This section describes how to compute the Mean Opinion Score (MOS) for audio quality.

git clone https://github.com/microsoft/DNS-Challenge
conda create -n mos python=3.8
conda activate mostest
conda install conda-forge::gcc=13.3.0
conda install conda-forge::libsndfile
pip install librosa, onnxruntime, pandas, tqdm

Following the instructions in the Github repository of DNS-Challenge/DNSMOS, you can then execute the relevant scripts to compute the MOS.

Multimodal Multi-Loss Fusion Network (MMML)

Computing the emotional precision off Moshi's answers requires the use of a multimodal emotion recognition system that supports both audio and text modalities. In this section, we describe how to install MMML:

git clone https://github.com/zehuiwu/MMML ../
conda create -n mmml python=3.8.10
conda activate mmml
pip install -r ../MMML/requirements.txt

Credits

This project exists thanks to the prior work of amazing scientists:

• Kyutai Team – composed of Patrick Pérez, Édouard Grave, Laurent Mazaré, Neil Zeghidour and Alexandre Défossez – who devised Moshi and wrote a related paper (Défossez et al., 2024);
• Zehui Wu, Ziwei Gong, Jaywon Koo and Julia Hirschberg who developed MMML and published a paper (Wu et al., 2023).

License

This work is MIT licensed and is based on existing open source projects.