Noise-Aware Audio Self-Supervised Learning (NA-SSL)

NA-SSL is a framework to extract clean SSL representations from noisy input by using auxiliary noise information. This repository contains the pretrained models and training recipes used in our IWAENC 2026 and DCASE 2026 workshop submissions, and our DCASE 2026 Task 2 Challenge submission.

If you use any part of this code for your work, we ask that you include at least one of the following citations:

@InProceedings{Fujimura2026IWAENC_nassl,
  author    =  {Fujimura, Takuya and Masuyama, Yoshiki and Wichern, Gordon and Boeddeker, Christoph and Richter, Julius and {Le Roux}, Jonathan},
  title     =  {{NABEATs}: Noise-Aware Audio Representation Learning},
  booktitle =	 {Proc. International Workshop on Acoustic Signal Enhancement (IWAENC)},
  year      =	 2026,
  month     =	 sep
}

@InProceedings{Fujimura2026DCASE_nassl,
  author    =  {Fujimura, Takuya and Wichern, Gordon and Masuyama, Yoshiki and Boeddeker, Christoph and Saijo, Kohei and Richter, Julius and Edo, Takahiro and {Le Roux}, Jonathan},
  title     =  {Anomalous Sound Detection Meets Noise-Aware Self-Supervised Learning},
  booktitle =	 {Proc. Workshop on Detection and Classification of Acoustic Scenes and Events (DCASE)},
  year      =	 2026,
  month     =	 oct
}

Supported Pretrained Models

The following pretrained checkpoints are provided through GitHub Releases.

Installation

Python 3.11 or later is required.

Pretrained model cache

Pretrained checkpoints are downloaded automatically from GitHub Releases and cached in ~/.cache/nassl.

export NASSL_CACHE_DIR=/path/to/cache

Inference

uv add git+https://github.com/merlresearch/noise-aware-audio-ssl.git

pip install git+https://github.com/merlresearch/noise-aware-audio-ssl.git

import torch

from nassl.utils.restore import restore_model

model = restore_model(model_id="dcase2026_nabeats_seed0", device="cpu")
model.eval()
x = torch.randn(1, 16000 * 10) # Input noisy (batch_size, time)
n_ref = torch.randn(1, 16000 * 10) # Reference noise (batch_size, time)
with torch.inference_mode():
    z = model(x, n_ref) # Estimate clean representation

model = restore_model(model_id="iwaenc2026_dbeats_seed0", device="cpu")
model.eval()
x = torch.randn(1, 16000 * 10) # Input noisy only
with torch.inference_mode():
    z = model(x)

model = restore_model(ckpt_path="/path/to/model.ckpt", device="cpu")

Training

git clone https://github.com/merlresearch/noise-aware-audio-ssl.git
cd noise-aware-audio-ssl
uv sync --extra train --locked

bash jobs/download/fsd50k.sh
bash jobs/download/wham.sh
bash jobs/download/demand.sh
bash jobs/download/qut_noise.sh

bash jobs/preprocessing/base_training_data.sh

bash jobs/preprocessing/iwaenc_2026.sh
bash jobs/train/iwaenc_2026.sh

bash jobs/preprocessing/dcase_2026.sh
bash jobs/train/dcase_2026.sh

By default, training results are saved in noise-aware-audio-ssl/scratch/result/. Pretrained checkpoints can be loaded using model = restore_model(ckpt_path="/path/to/model.ckpt"), as described in the inference section.

Contributing

See CONTRIBUTING.md for our policy on contributions.

Copyright and license

Released under AGPL-3.0-or-later license, as found in the LICENSE.md file.

All files, except as noted below:

Copyright (C) 2026 Mitsubishi Electric Research Laboratories (MERL)

SPDX-License-Identifier: AGPL-3.0-or-later

BEATs

The following files:

• src/nassl/model/beats/backbone.py
• src/nassl/model/beats/BEATs.py
• src/nassl/model/beats/modules.py
• src/nassl/model/beats/quantizer.py
• src/nassl/model/beats/Tokenizers.py

were taken from BEATs with only minor modifications to import statements.

Copyright (C) 2026 Mitsubishi Electric Research Laboratories (MERL)
Copyright (c) 2022 Microsoft

The following files:

• src/nassl/model/beats_extension/dbeats.py
• src/nassl/model/beats_extension/nabeats.py

were adapted from BEATs:

• Code source: BEATs
• License: MIT (local copy, upstream license)
  • Includes FAIRSEQ
    • License: MIT (local copy, upstream license)
  • Includes VQGAN
    • License: MIT (local copy, upstream license).

Copyright (C) 2026 Mitsubishi Electric Research Laboratories (MERL)
Copyright (c) 2022 Microsoft
Copyright (c) 2019 Facebook, Inc. and its affiliates.
Copyright (c) 2020 Patrick Esser and Robin Rombach and Björn Ommer

EAT

The following files:

• src/nassl/model/eat/tools.py
• src/nassl/model/eat_extension/naeat.py

were adapted from EAT:

• Code source: worstchan/EAT-base_epoch30_pretrain
• Original repository: cwx-worst-one/EAT
• License: MIT (local copy, upstream license)

Copyright (C) 2026 Mitsubishi Electric Research Laboratories (MERL)
Copyright (c) 2024 Wenxi Chen

Dasheng

The following file:

• src/nassl/model/dasheng_extension/nadasheng.py

was adapted from Dasheng:

• Code source: mispeech/dasheng-base
• Original repository: RicherMans/Dasheng
• License: Apache License 2.0 (local copy, upstream license)

Copyright (C) 2026 Mitsubishi Electric Research Laboratories (MERL)
Copyright (C) 2024 Xiaomi Corporation