TimbreTune

TimbreTune is a JUCE audio plugin (VST3 / AU / Standalone) for zero-shot vocal timbre conversion. You load a source vocal and a reference voice, tweak parameters, and convert. The plugin talks to a local Python server that runs the Seed-VC model; the server can be started by the plugin automatically, or you can run it yourself.

Installation

• Python 3.10 (Windows, Mac Apple Silicon, or Linux).

Dependencies:

# Windows / Linux
pip install -r requirements.txt

# Mac M Series (Apple Silicon)
pip install -r requirements-mac.txt

If you can’t access Hugging Face, use a mirror:

export HF_ENDPOINT=https://hf-mirror.com   # then run server/plugin as usual

The first time the server runs, the pretrained Seed-VC checkpoint and config are downloaded automatically from Hugging Face.

Usage

Running the server (optional)

The plugin starts the server by itself when you open it (via launch_server.sh). You don’t have to run anything first.

If you want to run the server yourself (e.g. to see logs in the terminal or to call the API from the command line):

./launch_server.sh

Or:

python server.py [--gpu 0] [--fp16] [--port 8765]

Keep it running while you use the plugin or the Gradio/CLI options below.

Building and running the plugin

From the project root:

cd juce-app
mkdir -p build && cd build
cmake ..
cmake --build . --target TimbreTune_Standalone   # or TimbreTune_VST3 / TimbreTune_AU
cd ..
cmake --build build --config Release # to build all

• Standalone: Run the built app. On macOS:

  open build/TimbreTune_artefacts/Standalone/MacOS/TimbreTune.app
  

  On Windows, run the .exe in the Standalone folder.
• VST3 / AU: Build the TimbreTune_VST3 or TimbreTune_AU target; the build copies the plugin into your system plugin folder (e.g. on Mac: ~/Library/Audio/Plug-Ins/VST3/ or ~/Library/Audio/Plug-Ins/Components/). Rescan plugins in your DAW and load TimbreTune on a track or bus.

In the plugin: load Source (vocal to convert) and Reference (target timbre), optionally Record from the DAW, adjust the parameters below, then Convert. Use Export to save the result.

Parameters you can tweak:

Optional: Gradio UI (browser, no plugin)

To try conversion in a browser without the plugin:

python app_svc.py [--checkpoint <path>] [--config <path>] [--fp16 True]

Then open http://localhost:7860/. Omit --checkpoint and --config to use the default pretrained model.

Optional: Command-line conversion (no plugin, no Gradio)

To run conversion from the console (e.g. for scripts or batch use), use the inference script in legacy_seedvc. From the project root:

PYTHONPATH=. python legacy_seedvc/inference.py --source <source.wav> --target <reference.wav> --output <output-dir> [options]

See legacy_seedvc/README.md for full options (diffusion steps, length adjust, CFG rate, pitch shift, etc.).

Project layout

• server.py – Flask API used by the plugin (POST/GET convert).
• app_svc.py – Seed-VC inference + Gradio UI; used by server.py for conversion.
• launch_server.sh – Script the plugin runs to start the server (or run it yourself).
• juce-app/ – JUCE plugin source (VST3, AU, Standalone).
• configs/, modules/, hf_utils.py, morph_utils.py – Required by app_svc and the model.
• legacy_seedvc/ – Original Seed-VC material (training, other UIs, CLI inference, evaluation docs). See legacy_seedvc/README.md.

Evaluation (proving the morph works)

To show that (1) the midpoint (α=0.5) is a true in-between of source and target, and (2) similarity to the target increases smoothly as α goes 0→1:

1. Create a pairs CSV with columns source_path,target_path,pair_id (or pair_id,source_path,target_path) and paths to WAVs.
2. Generate outputs at several α values (from the evaluation/ folder):

   cd evaluation
   python eval_morph.py --pairs eval_pairs.csv --out-dir eval_outputs
   

3. Compute metrics (optionally with a second embedding model) and plot (still inside evaluation/):

   python eval_metrics.py --eval-dir eval_outputs
   python eval_metrics.py --eval-dir eval_outputs --embedding-model ecapa
   python eval_plot.py --report eval_report.csv --report-ecapa eval_report_ecapa.csv --out eval_plot
   

   This produces eval_plot.png / eval_plot.pdf (similarity to source and target vs α) and all eval reports inside evaluation/. In MATLAB you can run eval_plot from the evaluation/ directory to get the same curve from eval_report.csv.

See EVALUATION.md for the full plan (baselines, metrics, optional WER and listening test).

Credits

Voice conversion model and inference code from Seed-VC (Plachta et al.). TimbreTune adds the JUCE plugin and the server.py backend.