🔥🔥🔥Nes2Net: A Lightweight Nested Architecture for Foundation Model Driven Speech Anti-spoofing

Official release of pretrained models and scripts for "Nes2Net: A Lightweight Nested Architecture for Foundation Model Driven Speech Anti-spoofing"

Accpeted to: IEEE Transactions on Information Forensics and Security (T-IFS)

IEEE Link: https://ieeexplore.ieee.org/document/11222612 arXiv Link: https://arxiv.org/abs/2504.05657

📢 [IMPORTANT] This repo is for the Controlled Singing Voice Deepfake Detection (CtrSVDD) dataset. For speech cases, please refer to following two repos:

For the ASVspoof 19&21 and In-the-Wild dataset: 👉 asvspoof19/21 & In-the-Wild
For the ASVspoof 5 dataset: 👉 asvspoof5

Update:

🔥[March, 2026] Nes2Net models are now avaiable on 🤗Hugging Face!!! 👉 🤗HuggingFace. Thanks to @nielsr

🔥[April, 2025] ASVspoof 5 pretrained model and codes are now avaiable!!! 👉 asvspoof5

Pretrained Models

• Only best model checkpoints are provided.

Usage:

Setup:

1. Git clone this repo.

2. Build the environment:

   conda env create -f SVDD.yml
   

   or

   pip install -r requirements.txt
   

   👉 You may need to adjust some library versions based on your CUDA version.

3. Set up S3PRL for the WavLM front-end by following this link: https://github.com/s3prl/s3prl

Easy inference with pre-ptrained models

If you want to perform easy inference with pretrained models:

1. Download the pretrained checkpoints from the table above via the provided Google Drive links (e.g., WavLM_Nes2Net_X_SeLU).

2. Run the following command:

   CUDA_VISIBLE_DEVICES=0 python easy_inference_demo.py \
   --model_path [pretrained_model_path] \
   --file_to_test [the file to test] \
   --model_name xxxx
   

   Example:

   CUDA_VISIBLE_DEVICES=0 python easy_inference_demo.py \
   --model_path "/data/tianchi/Nes2Net_SVDD_ckpts/WavLM_Nes2Net_X_SeLU_e74_seed420_valid0.04245662278274772.pt" \
   --file_to_test "/home/tianchi/data/SVDD2024/test_set/CtrSVDD_0115_E_0092590.flac" \
   --model_name WavLM_Nes2Net_X_SeLU
   

   Alternatively, to run inference using the CPU, set:

   CUDA_VISIBLE_DEVICES= 
   

3. Interpreting Prediction Scores When training on CtrSVDD, the target labels are:

   Important: For models trained on the CtrSVDD dataset:

   • 1 indicates real audio
   • 0 indicates fake audio

   During inference, the model outputs a continuous score. Here’s how to interpret it:

   • Scores > 0.8 → Highly likely to be real audio (≈95% of real samples)
   • Scores < 0 → Highly likely to be fake audio (≈95% of fake samples)
   • Scores between 0 and 0.8 → You can apply a decision threshold (e.g., based on EER) for classification.

   Example:
   A score of 3.4 means the model is confident the input is a real sample.

---

⚠️ Note: This repository is designed for singing voice anti-spoofing tasks. If you are working on speech-oriented detection, please refer to the official repositories of the ASVspoof dataset series above for more appropriate tools and models.

Train the model by yourself

If you want to train the model yourself, check the command template in: train.sh

Example Command:

python train.py --base_dir /home/tianchi/data/SVDD2024/ --algo 8 --gpu_id 2 --T_max 5 --epochs 75 --lr 0.000001 --batch_size 34 \
--agg SEA --pool_func 'mean' --dilation 1 --Nes_ratio 8 8 --SE_ratio 1 --model_name WavLM_Nes2Net_X --seed 420 \
--foldername WavLM_SEA_Nes2Net_X_mean_8x8_SEr1_dila1_algo8_Tmax5_bz34_lr1e6_seed420

• Change the --base_dir to match the path of your SVDD2024 dataset.
• The --foldername can be set according to your preference.

Test on the CtrSVDD dataset

If you want to test on the CtrSVDD dataset using the released pretrained models or your own trained model:

I. Testing inference

1. Use the command template in: eval.sh. Example Command:

CUDA_VISIBLE_DEVICES=6 python eval.py --base_dir /home/tianchi/data/SVDD2024/test_set \
--model_path "/data/tianchi/Nes2Net_SVDD_ckpts/WavLM_Nes2Net_X_e75_seed420_valid0.03192785031473534.pt" \
--agg SEA --pool_func 'mean' --dilation 1 --Nes_ratio 8 8 --SE_ratio 1 --model_name WavLM_Nes2Net_X \
--outputname E75_WavLM_SEA_Nes2Net_X_mean_8x8_SEr1_dila1_algo8_Tmax5_bz34_lr1e6_seed420

2. Modify the following parameters as needed:
   • --base_dir → Set this to the path of your SVDD2024 test set.
   • --model_path → Specify the path of the checkpoint to be tested.
     • The default path for a model trained using our script is: logs/[outputname]/[YYYYMMDD]-[6digits]/checkpoints/model_[epoch]_EER_[valid EER].pt
     • You should use the checkpoint with the smallest validation EER for testing.
     • It can also be a checkpoint downloaded from the Google Drive link above.
   • --agg --pool_func --dilation --Nes_ratio --SE_ratio --model_name → Set these to match your training configuration.
     • If you are using the pretrained model, these settings can be found in eval.sh.

II. Obtain EER and minDCF results

To compute the final Equal Error Rate (EER) and minimum Detection Cost Function (minDCF), as well as detailed results for each sub-trial, run:

python EER_minDCF.py --labels_file [path to the CtrSVDD test set label txt] \
--path [path to the score file generated by above command]

Example Command:

python EER_minDCF.py --labels_file '/home/tianchi/data/SVDD2024/test.txt' \
--path scores/E75_WavLM_SEA_Nes2Net_X_mean_8x8_SEr1_dila1_algo8_Tmax5_bz34_lr1e6_seed420.txt

Example output:

---------------------------------------------------------
dataset m4singer - EER: 2.4536%  minDCF: 0.024288
dataset kising - EER: 8.6851%  minDCF: 0.085662
---------------------------------------------------------
excluding A14 only, #: 67579
- EER: 2.2230%  minDCF: 0.022174
---------------------------------------------------------
excluding both acesinger and A14, #: 64734
- EER: 2.4782%  minDCF: 0.024745
(atkID A09) - EER: 1.2288%  minDCF: 0.011929
(atkID A10) - EER: 0.6305%  minDCF: 0.006173
(atkID A11) - EER: 2.0893%  minDCF: 0.018279
(atkID A12) - EER: 5.2686%  minDCF: 0.051162
(atkID A13) - EER: 0.8284%  minDCF: 0.008284
---------------------------------------------------------

Reference Repo

Thanks for following open-source projects:

1. wav2vec2 + AASIST & Rawboost: https://github.com/TakHemlata/SSL_Anti-spoofing Paper: [model], [Rawboost]
2. SEA aggregation: https://github.com/Anmol2059/SVDD2024 Paper: [SEA]
3. AttM aggregation: https://github.com/pandarialTJU/AttM_INTERSPEECH24 Paper: [AttM]
4. WavLM pretrained model is from S3PRL: https://github.com/s3prl/s3prl

License and Usage

This project is released under the Apache License 2.0. The authors welcome and encourage academic use of this project and its released models, and are friendly to commercial use, modification, and redistribution, subject to the terms of the Apache License 2.0. If you use our models or code in your research or applications, we kindly ask that you acknowledge this work and cite the associated paper.

This project may rely on or incorporate publicly available models, datasets, and third-party open-source software. Users are solely responsible for reviewing and complying with the licenses and terms applicable to these third-party components. The release of this project does not grant any additional rights to third-party models, datasets, or software, and the authors are not responsible for users' compliance with such third-party terms.

The project, code, and models are provided "AS IS", without warranties of any kind. To the extent permitted by applicable law, the authors shall not be liable for any claims, damages, or other liabilities arising from the use, modification, or redistribution of the project or models.

Cite

@ARTICLE{Nes2Net,
  author={Liu, Tianchi and Truong, Duc-Tuan and Das, Rohan Kumar and Lee, Kong Aik and Li, Haizhou},
  journal={IEEE Transactions on Information Forensics and Security}, 
  title={Nes2Net: A Lightweight Nested Architecture for Foundation Model Driven Speech Anti-Spoofing}, 
  year={2025},
  volume={20},
  number={},
  pages={12005-12018},
  keywords={Foundation models;Feature extraction;Computational modeling;Computer architecture;Computational efficiency;Dimensionality reduction;Acoustics;Kernel;Robustness;Deepfakes;Deepfake detection;speech anti-spoofing;Res2Net;Nes2Net;SSL;speech foundation model},
  doi={10.1109/TIFS.2025.3626963}}