Speech-to-Speech: Darija Conversational AI

This project is a full speech-to-speech conversational pipeline for Moroccan Darija. It transcribes speech, generates a natural language response, and synthesizes the reply as speech—all in Darija. The system uses state-of-the-art models for speech recognition, language generation, and text-to-speech.

Overview

• Speech-to-Text (STT): Fine-tuned Whisper model for Darija transcription.
• Language Model (LLM): Arabic conversational LLM (AceGPT-7B or fallback to AraGPT2).
• Text-to-Speech (TTS): DarijaTTS model for high-quality speech synthesis.
• Web Interface: Simple frontend for uploading audio and listening to responses.

Setup Instructions

1. Clone the repository:

   git clone https://github.com/imranboutadghart/speech-to-speech.git
   cd speech-to-speech
   

2. Install dependencies: Ensure you have Python 3.8+ and pip installed.

   pip install -r requirements.txt
   

3. Run the server:

   python app.py
   

4. Open the web interface: Open index.html in your browser or access the server at http://localhost:5000 if running locally.

Usage

1. Record or upload a Darija audio file (supported: wav, mp3, ogg, m4a, flac).
2. The backend transcribes, generates a response, and returns synthesized speech.
3. Listen to the AI-generated reply in Darija.

Project Structure

• app.py — Flask backend for the speech-to-speech pipeline
• index.html — Web frontend for user interaction
• requirements.txt — Python dependencies
• finetunes/ — Notebooks for model fine-tuning
  • whisper.ipynb — Fine-tuning Whisper for Darija STT
  • outetts.ipynb — Fine-tuning Outetts for Darija TTS
• outputs/ — Example output audio files

Notes

• For best results, use a machine with a CUDA-capable GPU.
• The TTS model uses outetts==0.3.3.