Hindi to Tamil Voice Translator

A complete Speech-to-Speech translation pipeline that records spoken Hindi and translates it into spoken Tamil. This project leverages open-source AI models for Automatic Speech Recognition (ASR), Machine Translation (MT), and Text-to-Speech (TTS).

Features

• Voice Recording: Captures live audio input via your microphone.
• Hindi Speech Recognition (ASR): Powered by faster-whisper for fast, accurate transcription.
• Machine Translation: Uses AI4Bharat's IndicTrans2 (ai4bharat/indictrans2-indic-indic-1B) for high-quality Hindi to Tamil translation.
• Tamil Text-to-Speech (TTS): Uses Piper TTS with the ta_IN-Valluvar-medium.onnx model for natural-sounding Tamil voice synthesis.

Requirements

The project relies on Python 3 and the following core libraries:

• faster-whisper
• transformers
• sentencepiece
• torch
• sounddevice
• scipy
• soundfile

Note: The translation step also uses IndicNLP (indic-nlp-library) for transliteration.

Installation

1. Navigate to the project directory:

   cd path/to/HindiTamilTranslator
   

2. Activate your virtual environment (e.g., venv, venv311, or venv313):

   venv\Scripts\activate
   

3. Install the dependencies:

   pip install -r requirements.txt
   pip install indic-nlp-library
   

4. Ensure Piper is Set Up: The TTS step expects the Piper executable to be located at .\piper\piper\piper.exe and the Tamil voice model (ta_IN-Valluvar-medium.onnx) to be present inside the .\models\ directory.

Usage

You can run the entire pipeline at once using the main application script:

python app.py

How the Pipeline Works (app.py)

When you run app.py, it sequentially executes four steps:

1. record.py: Prompts you to speak in Hindi. It records audio for 5 seconds and saves it as recorded.wav.
2. asr.py: Reads recorded.wav, transcribes the Hindi speech using Whisper, and saves the recognized text to output.txt.
3. translate.py: Reads the Hindi text from output.txt, translates it into Tamil using IndicTrans2, and saves the translated text to translation.txt.
4. tts.py: Reads the Tamil text from translation.txt and uses Piper TTS to generate the final synthesized audio, saved as tamil_output.wav.

Running Individual Modules

You can also run each step manually for testing purposes. Just ensure you run them in order:

python record.py
python asr.py
python translate.py
python tts.py

Evaluation

This repository also contains several scripts dedicated to evaluating translation quality against your dataset, such as:

• chrf_evaluation.py: Computes the chrF score for translations.
• comet_evaluation.py: Uses the COMET framework for semantic evaluation.
• semantic_evaluation.py & evaluation.py: Additional evaluation pipelines to assess model performance and output results to CSVs.

Key Files

• app.py: The main entry point orchestrating the pipeline.
• record.py, asr.py, translate.py, tts.py: Core modules for each step in the S2S pipeline.
• requirements.txt: Project dependencies.
• dataset.csv: Evaluation dataset.
• models/: Contains the local ONNX models.
• piper/: Contains the Piper TTS binaries.