ReCLAP: Improving Zero Shot Audio Classification by Describing Sounds

This is the official implementation of our paper ReCLAP: Improving Zero Shot Audio Classification by Describing Sounds.

Training data can be dowloaded from ReCLAP's Google Drive. Model checkopoint can be dowloaded from here.

---

Setup 🏋️

conda create -n reclap python=3.10
conda activate reclap
cd ReCLAP
pip install -r requirements.txt

Training 🏃‍♂️

Update the path to training csv (--train-data) validation csv (--val-data) and pretrained HTSAT checkpoint (--pretrained-audio) in run.sh file. The csvs can be downloaded from ReCLAP's Google Drive.

To run the training:

cd ReCLAP/train/src/laion_clap
sh run.sh

Prompt Augmentations

More details on prompt augemntation can be found in prompts.

Acknowledgement 🌻

We would like to thank the authors of LAION-CLAP for open-sourcing their code, which inspired our work.