End-to-End Supervised Hierarchical Graph Clustering for Speaker Diarization

This is the implementation of the following papers:

• Singh, Prachi, Ganapathy, Sriram, "End-to-End Supervised Hierarchical Graph Clustering for Speaker Diarization", 2024 (paper)

• Singh, Prachi, Kaul Amrit (2023)."Supervised Hierarchical Clustering using Graph Neural Networks for Speaker Diarization." Proceedings of ICASSP 2023. (paper)

• 24-04-2024 : Updated ReadMe and added missing directories

Overview

• Prerequisites
• Installation
• Running the recipes

• Cite
• Contact

Prerequisites

The following packages are required to run the code.

• Python >= 3.7
• Kaldi
• dscore
• Voxconverse
• AMI
• pyannote 2.1

Pretrained Models

The following pretrained models are provided.

• ETDNN x-vector model.
• PLDA models for Voxconverse and AMI dataset.
• SHARC models for Voxconverse and AMI.
• E-SHARC models for Voxconverse and AMI.

Installation

Clone the repo and create a new virtual environment

• clone the repo:

$ git clone -b ESHARC git@github.com:prachiisc/SHARC.git
$ cd SHARC

• Create the environment: We recommend running the recipes from a fresh virtual environment. Make sure to activate the environment before proceeding.

$ conda create --name SHARC --file requirements.txt
$ conda activate SHARC

• It is recommended to install pyannote in different environment to avoid clashes.

• Install Kaldi. If you are a Kaldi novice, please consult the following for additional documentation:

  • Kaldi tutorial
  • Kaldi for Dummies tutorial

• Go to cloned repository and create Softlinks of necessary directories given as:

$ local_dir="Full_path_of_cloned_repository"
$ KALDI_PATH=/path_of_kaldi_directory/kaldi
$ cd $local_dir
$ ln -sf $KALDI_PATH kaldi
$ . ./path.sh
$ ln -sf kaldi/egs/wsj/s5/steps .  # steps dir

• Check the data directories in tools_diar/data.
  Change tools_diar/data/<datasetname>/wav.scp with your path of wavfiles.

Running the E-SHARC recipes

We include full recipes for reproducing the results for Voxconverse and AMI dataset:

Testing on the AMI dataset

The following scripts perform Speech activity detection (SAD/VAD), X-vector extraction, E-SHARC, E-SHARC-Ovp, VBx in different stages.

   bash scripts/test_ami_main.sh

Testing on the Voxconverse dataset

The following scripts perform Speech activity detection (SAD/VAD), X-vector extraction, E-SHARC, E-SHARC-Ovp, VBx in different stages.

   bash scripts/test_vox_main.sh

Running the SHARC recipes

We include full recipes for reproducing the results for Voxconverse and AMI dataset:

Testing on the Voxconverse dataset

Step 1: X-vector extraction, groundtruth label creation and lists directory formation

   bash services/test_xvec_preprocess.sh <vox_set> nj

<vox_set> : vox_diar/vox_diar_test

nj : number of jobs [min(40,number of processors available)]

Step 2: Testing

   bash scripts/test_xvec_parallel.sh Vox

Testing on the AMI dataset

Step 1: X-vector extraction, groundtruth label creation and lists directory formation

   bash services/test_xvec_preprocess.sh <ami_set> nj

<ami_set> : ami_dev/ami_eval

nj : number of jobs [min(15,number of processors available)]

Step 2: Testing the SHARC model

   bash scripts/test_xvec_parallel.sh AMI

Training the model

Step 1: X-vector extraction, groundtruth label creation and lists directory formation

   bash services/test_xvec_preprocess.sh <train_set> <nj>

Step 2: Training for Voxconverse/AMI

   bash scripts/train_xvec.sh <Vox/AMI>

Cite

If you are using the resource, please cite as follows:

@article{singh2024overlap,
  title={End-to-End Supervised Hierarchical Graph Clustering for Speaker Diarization},
  author={Singh, Prachi and Ganapathy, Sriram},
  journal={arXiv preprint arXiv:2401.12850v2},
  year={2024}
}

@INPROCEEDINGS{10095372,  
  author={Singh, Prachi and Kaul, Amrit and Ganapathy, Sriram},
  booktitle={IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)},  
  title={Supervised Hierarchical Clustering Using Graph Neural Networks for Speaker Diarization},  
  year={2023}, 
  volume={}, 
  pages={1-5}, 
  doi={10.1109/ICASSP49357.2023.10095372}}
  

Contact

If you have any comment or question, please contact prachisingh@iisc.ac.in