EchoVerse - AI-Powered English Learning for Children

---

📖 Table of Contents

• Overview
• Features
• Demo
• Technology Stack
• Architecture
• Installation
  • Prerequisites
  • Flutter App Setup
  • Backend Setup
• Configuration
• Usage
• Project Structure
• AI Speech Module
• Contributing
• License
• Acknowledgments

---

Overview

EchoVerse is a comprehensive English language learning platform designed specifically for children aged 6-12. It combines cutting-edge AI technology with engaging educational content to create an immersive learning experience that makes language acquisition fun and effective.

Problem Statement

Traditional language learning methods often fail to engage young learners, relying on rote memorization and lacking real-time conversational practice. EchoVerse addresses these challenges by providing:

• Real-time AI Conversation: Practice speaking with an AI companion that understands context and provides instant feedback
• Personalized Learning: Adaptive content that adjusts to each child's proficiency level and learning pace
• Gamified Experience: Interactive games, stories, and activities that make learning enjoyable
• Safe Environment: Child-friendly content with parental controls and progress monitoring

Alignment with SDGs

EchoVerse contributes to the United Nations Sustainable Development Goals:

• SDG 4 - Quality Education: Provides accessible, personalized language learning
• SDG 10 - Reduced Inequalities: Makes English education available to diverse backgrounds
• SDG 8 - Decent Work: Equips children with foundational skills for future employment

---

✨ Features

Learning Modules

1. Letter Land - Alphabet Learning

• Interactive alphabet exploration with phonics
• Letter tracing and pronunciation practice
• Visual and audio learning aids
• Progress tracking for each letter

2. Wordy Woods - Vocabulary Building

• 24 themed topics (Animals, Food, Sports, etc.)
• 3 difficulty levels per topic
• Interactive flashcards with images and audio
• Spaced repetition for retention

3. Story Reading - Comprehension & Fluency

• 4 story categories with age-appropriate content
• Interactive quizzes after each story
• Text-to-speech narration
• Comprehension assessment

4. Educational Games

• Memory Match: Vocabulary reinforcement
• Sentence Builder: Grammar practice
• Word Puzzles: Spelling and recognition

🤖 AI TalkMate - Conversational AI

Real-time voice conversations powered by GPU-accelerated AI:

• Three Conversation Modes:

  • Free Chat: Open-ended conversations
  • Topic Talk: Focused discussions on specific themes
  • Pretend Play: Role-playing scenarios (restaurant, shopping, etc.)

• Two AI Providers:

  • Local (Whisper + Qwen): GPU-accelerated local models
  • Voxtral: Mistral API with enhanced capabilities

• Real-time Features:

  • Speech-to-Text (Whisper)
  • Natural Language Processing (Qwen2/Mistral)
  • Grammar & Fluency Analysis
  • Text-to-Speech (Piper/Kokoro)
  • Pronunciation Assessment

• Animated Avatars: GIF-based avatars with speaking/idle states synchronized with audio

📊 Progress Tracking & Analytics

• Session History: Detailed logs of all learning activities
• Performance Metrics:
  • Grammar scores
  • Fluency ratings
  • Pronunciation accuracy
  • Vocabulary growth
• Visual Reports: Charts and graphs for parents
• Achievement System: Badges and milestones

Parental Controls

• 5-Step Profile Creation:

  1. Basic information (name, age)
  2. Avatar selection
  3. Language preferences
  4. Interests and hobbies
  5. Learning goals & parental PIN

• Parental Dashboard:

  • Usage monitoring
  • Progress reports
  • Time limit controls
  • Content filtering

User Experience

• Beautiful UI: Modern, colorful design optimized for children
• Responsive Design: Works on phones, tablets, and web
• Offline Support: Core features available without internet
• Multi-language: Support for multiple interface languages
• Accessibility: Screen reader support, adjustable text sizes

---

🎬 Demo

Screenshots

Video Demo

Click the image above to watch the full demo on YouTube

---

Admin Dashboard

EchoVerse includes a separate Admin Dashboard for managing users, monitoring analytics, and controlling content.

Repository: echoverse_admin_dashboard

Features:

• User management and monitoring
• Learning analytics and reports
• Content moderation
• System configuration
• Session tracking

---

🛠 Technology Stack

Frontend (Mobile App)

Backend (AI Speech Module)

Infrastructure

• Hosting: Modal GPU (NVIDIA T4)
• Database: Firebase (Firestore + Realtime DB)
• Storage: Firebase Storage
• Authentication: Firebase Auth
• Deployment: Modal Serverless

---

🏗 Architecture

System Architecture

graph TB
    subgraph "Mobile App (Flutter)"
        UI[User Interface]
        SM[State Management<br/>Provider]
        WS[WebSocket Service]
        TTS[TTS Service]
        AR[Audio Recording]
        AP[Audio Player]
    end

    subgraph "Firebase"
        AUTH[Authentication]
        FS[Firestore]
        RT[Realtime DB]
        STOR[Storage]
    end

    subgraph "AI Backend (Modal GPU)"
        API[FastAPI Server]
        STT[Whisper STT]
        LLM[Qwen2 LLM]
        GRAM[T5 Grammar]
        TTS_B[Piper TTS]
        PIPE[Processing Pipeline]
    end

    UI --> SM
    SM --> WS
    SM --> TTS
    SM --> AR
    SM --> AP
    
    WS <--> API
    AR --> WS
    AP <-- WS
    
    UI --> AUTH
    SM --> FS
    SM --> RT
    UI --> STOR
    
    API --> STT
    API --> LLM
    API --> GRAM
    API --> TTS_B
    API --> PIPE

Data Flow - AI Conversation

sequenceDiagram
    participant User
    participant Flutter
    participant WebSocket
    participant Backend
    participant AI Models

    User->>Flutter: Speaks into microphone
    Flutter->>WebSocket: Stream audio chunks
    WebSocket->>Backend: Binary audio data
    Backend->>AI Models: Whisper STT
    AI Models-->>Backend: Transcript
    Backend->>WebSocket: Send transcript
    WebSocket->>Flutter: Display user message
    
    Backend->>AI Models: Qwen2 LLM
    AI Models-->>Backend: Stream response tokens
    Backend->>WebSocket: Stream tokens
    WebSocket->>Flutter: Display AI response
    
    Backend->>AI Models: Piper TTS
    AI Models-->>Backend: Audio chunks
    Backend->>WebSocket: Binary audio
    WebSocket->>Flutter: Play audio
    Flutter->>User: Hears AI response
    
    Backend->>AI Models: Grammar analysis
    AI Models-->>Backend: Feedback
    Backend->>WebSocket: Send feedback
    WebSocket->>Flutter: Display metrics

---

📥 Installation

Prerequisites

• Flutter SDK: 3.2.3 or higher (Install Flutter)
• Dart SDK: 3.2.3 or higher (included with Flutter)
• Python: 3.11 or higher (Install Python)
• Firebase Account: Create Firebase Project
• Modal Account: Sign up for Modal (for AI backend)
• Git: For version control

Flutter App Setup

1. Clone the repository:

   git clone https://github.com/r-zeeshan/echoverse-v2.git
   cd echoverse-v2
   

2. Install Flutter dependencies:

   flutter pub get
   

3. Configure Firebase:

   • Create a new Firebase project at Firebase Console

   • Add Android and iOS apps to your Firebase project

   • Download google-services.json (Android) and GoogleService-Info.plist (iOS)

   • Place files in respective directories:

     • Android: android/app/google-services.json
     • iOS: ios/Runner/GoogleService-Info.plist

   • Enable Firebase services:

     • Authentication (Email/Password, Google Sign-In)
     • Cloud Firestore
     • Realtime Database
     • Storage

4. Set up Firebase Rules:

   • See FIREBASE_RULES.md for configuration details
   • Copy rules from FIREBASE_REALTIME_DATABASE_RULES.json

5. Run the app:

   # For Android
   flutter run -d android
   
   # For iOS
   flutter run -d ios
   
   # For Web
   flutter run -d chrome
   

Backend Setup

Option 1: Modal GPU Deployment (Recommended for Production)

1. Install Modal CLI:

   pip install modal
   

2. Authenticate with Modal:

   modal token new
   

3. Configure environment:

   cd speech-module
   cp .env.modal.example .env.modal
   # Edit .env.modal with your API keys
   

4. Deploy to Modal:

   modal deploy modal_app.py
   

5. Get deployment URL:

   modal app list
   # Copy the WebSocket URL
   

6. Update Flutter app configuration:

   • Edit lib/config/app_config.dart
   • Update modalWebSocketUrl with your Modal deployment URL

Cost: ~$0.40/hour | Free Credits: $30 (covers 75 hours)

Option 2: Local Development

1. Create virtual environment:

   cd speech-module
   python -m venv venv
   
   # Windows
   venv\Scripts\activate
   
   # macOS/Linux
   source venv/bin/activate
   

2. Install dependencies:

   pip install -r requirements.txt
   

3. Download AI models (optional for local testing):

   # Models are automatically downloaded on first run
   # Or manually download from Hugging Face
   

4. Run local server:

   uvicorn app.main:app --host 0.0.0.0 --port 9000 --reload
   

5. Update Flutter app:

   • Edit lib/config/app_config.dart
   • Set useModalGPU = false
   • Local URL: ws://localhost:9000/ws/speech-session-streaming

Note: Local development runs on CPU (3-4x slower than GPU)

---

⚙️ Configuration

Flutter App Configuration

Edit lib/config/app_config.dart:

class AppConfig {
  // Backend Configuration
  static const bool useModalGPU = true;  // true for production, false for local
  
  // Provider Configuration
  static const String defaultProvider = 'local';  // 'local' or 'voxtral'
  
  // Modal GPU URLs (Production)
  static const String modalWebSocketUrl = 'wss://YOUR-MODAL-URL/ws/speech-session-streaming';
  static const String modalHttpUrl = 'https://YOUR-MODAL-URL';
  
  // Local Backend (Development)
  static const String backendHost = 'localhost';
  static const int backendPort = 9000;
}

Backend Configuration

Edit speech-module/.env.modal:

# Device Configuration
DEVICE=cuda
COMPUTE_TYPE=float16

# Model Configuration
WHISPER_MODEL_SIZE=small
LLM_MODEL_PATH=./models/Llama-3.2-3B-Instruct-Q8_0.gguf
TTS_MODEL_PATH=./models/en_US-amy-medium.onnx

# GPU Settings
GPU_MEMORY_FRACTION=0.8
N_GPU_LAYERS=35

# Mistral API (for Voxtral provider)
MISTRAL_API_KEY=your_mistral_api_key_here

---

🚀 Usage

For Children

1. Create Profile: Parents set up child's profile with age, interests, and learning goals

2. Choose Activity:

   • Letter Land: Learn alphabet and phonics
   • Wordy Woods: Build vocabulary
   • Story Time: Read interactive stories
   • Games: Play educational games
   • AI TalkMate: Practice conversation

3. Track Progress: View achievements and learning milestones

For Parents

1. Monitor Progress: Access detailed reports on child's learning
2. Set Goals: Define learning objectives and time limits
3. Review Sessions: See conversation history and feedback
4. Adjust Settings: Control content access and difficulty levels

For Developers

1. Run Tests:

   flutter test
   

2. Build for Production:

   # Android
   flutter build apk --release
   
   # iOS
   flutter build ios --release
   
   # Web
   flutter build web --release
   

3. Monitor Backend:

   # Modal logs
   modal app logs echoverse-speech-module --follow
   
   # Check GPU usage
   modal profile current
   

---

📁 Project Structure

echoverse-v2/
├── android/                    # Android native code
├── ios/                        # iOS native code
├── web/                        # Web deployment files
├── windows/                    # Windows desktop support
├── linux/                      # Linux desktop support
├── macos/                      # macOS desktop support
├── lib/                        # Flutter application code
│   ├── config/                 # App configuration
│   ├── models/                 # Data models
│   │   ├── chat_message.dart
│   │   ├── chat_session.dart
│   │   ├── user_progress.dart
│   │   └── ...
│   ├── providers/              # State management
│   │   └── chat_provider.dart
│   ├── screens/                # UI screens (31 screens)
│   │   ├── home_screen.dart
│   │   ├── letter_detail_screen.dart
│   │   ├── ai_talk_mate_screen_realtime.dart
│   │   ├── story_reading_screen.dart
│   │   └── ...
│   ├── services/               # Business logic (15 services)
│   │   ├── auth_service.dart
│   │   ├── websocket_service.dart
│   │   ├── audio_player_service.dart
│   │   ├── audio_recording_service.dart
│   │   ├── session_storage_service.dart
│   │   └── ...
│   ├── widgets/                # Reusable UI components
│   │   ├── profile_creation/
│   │   └── ...
│   ├── utils/                  # Utility functions
│   └── main.dart               # App entry point
├── assets/                     # Static assets
│   ├── images/                 # Images and icons
│   ├── avatars/                # Avatar GIFs
│   ├── audio/                  # Audio files
│   ├── data/                   # CSV data files
│   ├── alphabet_dataset.csv
│   ├── worldy_words_dataset.csv
│   └── stories/                # Story JSON files
│       ├── islamic_stories.json
│       ├── animal_stories.json
│       ├── fairy_tales.json
│       └── adventure_stories.json
├── speech-module/              # AI Backend
│   ├── app/
│   │   ├── api/                # API endpoints
│   │   │   └── v1/
│   │   │       └── websocket_streaming.py
│   │   ├── services/           # AI services
│   │   │   ├── stt_service.py
│   │   │   ├── tts_service.py
│   │   │   ├── llm_service.py
│   │   │   ├── voxtral_service.py
│   │   │   └── pipeline.py
│   │   └── main.py
│   ├── modal_app.py            # Modal deployment config
│   ├── requirements.txt        # Python dependencies
│   └── README.md               # Backend documentation
├── diagrams/                   # Architecture diagrams
├── pubspec.yaml                # Flutter dependencies
├── analysis_options.yaml       # Dart analyzer configuration
├── .gitignore                  # Git ignore rules
├── .metadata                   # Flutter metadata
├── README.md                   # This file
├── PROJECT_INDEX.md            # Detailed project documentation
├── INITIAL_PROPOSAL.md         # FYP proposal
├── PROGRESS_REPORT.md          # Development progress
├── FIREBASE_RULES.md           # Firebase rules documentation
└── FIREBASE_REALTIME_DATABASE_RULES.json  # Firebase rules config

---

🎙 AI Speech Module

The AI Speech Module is a GPU-accelerated backend that powers real-time conversations. See speech-module/README.md for detailed documentation.

Key Components

1. Speech-to-Text (STT):

   • Whisper small model
   • Optimized for children's voices
   • Real-time transcription

2. Language Model (LLM):

   • Qwen2-1.5B (local) or Mistral API (Voxtral)
   • Child-friendly prompting
   • Context-aware responses

3. Grammar & Fluency Analysis:

   • T5-base for grammar correction
   • Custom fluency scoring
   • Pronunciation assessment

4. Text-to-Speech (TTS):

   • Piper TTS (fast, efficient)
   • Kokoro TTS (high quality)
   • Natural-sounding voices

Performance

• GPU (Modal T4): ~600ms total pipeline latency
• CPU (Local): ~2000ms total pipeline latency
• 3-4x speedup with GPU acceleration

API Endpoints

WebSocket:

ws://localhost:9000/ws/speech-session-streaming?
  user_id=<user_id>&
  child_name=<name>&
  age=<age>&
  hobbies=<hobbies>&
  mode=<free|topic|roleplay>&
  provider=<local|voxtral>

REST API (Testing):

• POST /api/transcribe - Speech-to-Text
• POST /api/test/stt - Test STT
• POST /api/test/tts - Test TTS

---

🤝 Contributing

We welcome contributions! Please follow these guidelines:

1. Fork the repository
2. Create a feature branch: git checkout -b feature/amazing-feature
3. Commit changes: git commit -m 'Add amazing feature'
4. Push to branch: git push origin feature/amazing-feature
5. Open a Pull Request

Development Guidelines

• Follow Flutter/Dart style guide
• Write meaningful commit messages
• Add tests for new features
• Update documentation as needed
• Ensure code passes linting: flutter analyze

---

📄 License

This project is licensed under the MIT License - see the LICENSE file for details.

---

🙏 Acknowledgments

Supervisor

• Ms. Amna Rauf - Department of Computer Science, NUML Islamabad

Technologies & Libraries

• Flutter - UI framework
• Firebase - Backend services
• Modal - GPU serverless platform
• OpenAI Whisper - Speech recognition
• Qwen2 - Language model
• Mistral AI - Voxtral API
• Piper TTS - Text-to-speech

Inspiration

• Duolingo ABC
• Rosetta Stone Kids
• HelloTalk for Kids

---

📞 Contact

Developers

Zeeshan Rajpoot

• 📧 Email: [email protected]
• 💻 GitHub: @r-zeeshan
• 💼 LinkedIn: Zeeshan Rajpoot

Umar Faizan

• 📧 Email: [email protected]
• 💻 GitHub: @u-faizan
• 💼 LinkedIn: Umar Faizan

Project Repository: https://github.com/r-zeeshan/echoverse-v2

---

📚 Additional Documentation

• Project Index - Comprehensive project overview
• Initial Proposal - FYP proposal document
• Progress Report - Development progress
• Firebase Setup Guide - Firebase configuration
• Speech Module Docs - AI backend documentation
• API Documentation - Complete API reference

---