HeartLab VR AI Integration System

Comprehensive Unity VR application for heart anatomy education with AI assistant integration, optimized for Meta Quest and featuring Turkish language medical terminology support.

🎯 Project Overview

HeartLab VR AI is an immersive virtual reality application that revolutionizes medical education by combining:

• Interactive 3D Heart Model with anatomical precision
• AI-Powered Medical Assistant using advanced language models
• Turkish Language Support for medical terminology
• Voice Recognition & Text-to-Speech for natural interaction
• Gaze Tracking & Hand Tracking for intuitive VR controls
• Progressive Learning Scenarios for educational effectiveness

✨ Key Features

🔬 Advanced VR Interactions

• Gaze-Based Selection: Look at heart parts to learn about them
• Hand Tracking Support: Natural hand gestures for interaction
• Voice Commands: Ask questions in Turkish about anatomy
• Haptic Feedback: Tactile responses for enhanced immersion
• 90 FPS Performance: Optimized for Meta Quest devices

🤖 AI Integration

• Hugging Face API: PubMedBERT/ClinicalBERT models for medical accuracy
• Context-Aware Responses: AI considers currently selected heart parts
• Turkish Medical Terminology: Comprehensive medical vocabulary support
• Response Caching: Improved performance and offline capabilities
• Fallback System: Educational content available without internet

❤️ Heart Anatomy System

• Interactive 3D Model: Detailed heart with all major structures
• Anatomical Mapping: Chambers, valves, vessels, and muscle tissue
• Real-time Highlighting: Visual feedback for part selection
• Educational Scenarios: Guided learning experiences
• Progressive Complexity: Adaptive learning difficulty

🎤 Speech & Audio

• Azure Speech Services: High-quality speech recognition and synthesis
• Turkish Voice Support: Native Turkish TTS with medical pronunciation
• Real-time Processing: < 2 second response times
• Audio Caching: Improved performance for repeated content
• Noise Filtering: Optimized for VR environment audio

🏗️ Technical Architecture

HeartLab VR AI/
├── Assets/
│   ├── Scripts/
│   │   ├── AI/                     # AI Integration
│   │   │   ├── HuggingFaceAPI.cs
│   │   │   ├── MedicalPromptManager.cs
│   │   │   ├── TurkishMedicalTerms.cs
│   │   │   └── ResponseCache.cs
│   │   ├── VR/                     # VR Interactions
│   │   │   ├── VRInputManager.cs
│   │   │   ├── GazeTracker.cs
│   │   │   ├── VoiceRecognition.cs
│   │   │   └── TextToSpeech.cs
│   │   ├── HeartModel/             # 3D Heart System
│   │   │   ├── HeartPartManager.cs
│   │   │   ├── AnatomicalMapping.cs
│   │   │   └── HeartPartDetector.cs
│   │   └── UI/                     # User Interface
│   │       ├── VRUIManager.cs
│   │       └── EducationalPanel.cs
│   ├── Prefabs/                    # Unity Prefabs
│   │   ├── HeartModel/
│   │   ├── VR/
│   │   └── UI/
│   └── Resources/                  # Data & Assets
│       ├── MedicalData/
│       └── Audio/
├── ProjectSettings/                # Unity Configuration
└── Documentation/                  # Project Documentation

🚀 Quick Start

Prerequisites

• Unity 2022.3 LTS or later
• Meta Quest headset or compatible VR device
• Azure Speech Services account (optional)
• Hugging Face API key (optional)

Installation

1. Clone the Repository

git clone https://github.com/aydincollab/HeartLab-VR-AI.git
cd HeartLab-VR-AI

2. Open in Unity

# Open Unity Hub and add the project folder
# Or open directly with Unity 2022.3 LTS

3. Configure API Keys (Optional)

# Set environment variables
export AZURE_SPEECH_KEY="your_azure_speech_key"
export AZURE_SPEECH_REGION="westeurope"
export HUGGINGFACE_API_KEY="your_huggingface_api_key"

4. Build for VR

# File > Build Settings
# Select Android platform for Meta Quest
# Configure XR settings for your VR headset

Quick Configuration

1. VR Setup: Enable XR Management and configure Oculus/OpenXR providers
2. AI Setup: Add API keys to script components or environment variables
3. Audio Setup: Configure microphone permissions for voice recognition
4. Build: Deploy to Meta Quest or run in VR-capable PC

🎮 Usage Examples

Basic VR Interaction

// Get VR input manager
var vrInput = VRInputManager.Instance;

// Listen for part selection
vrInput.OnHandPointingStarted += (position) => {
    Debug.Log($"User pointing at: {position}");
};

// Handle voice commands
vrInput.OnVoiceCommandDetected += (command) => {
    Debug.Log($"Voice command: {command}");
};

AI Query Integration

// Send medical query to AI
var aiAPI = HuggingFaceAPI.Instance;
aiAPI.SendMedicalQuery("Sol ventrikül ne işe yarar?", "sol_ventrikul");

// Handle AI response
aiAPI.OnResponseReceived += (response) => {
    textToSpeech.Speak(response);
};

Heart Part Management

// Select heart part
var heartManager = HeartPartManager.Instance;
var leftVentricle = heartManager.GetPartByName("sol_ventrikul");
heartManager.SelectPart(leftVentricle);

// Get anatomical information
var mapping = GetComponent<AnatomicalMapping>();
string info = mapping.GetDetailedDescription("sol_ventrikul", "tr");

🔧 Configuration

Environment Variables

# Required for AI features
AZURE_SPEECH_KEY=your_azure_speech_key_here
AZURE_SPEECH_REGION=westeurope
HUGGINGFACE_API_KEY=your_huggingface_api_key_here

Unity Settings

• Target Platform: Android (Meta Quest) / PC (Desktop VR)
• XR Providers: Oculus XR Plugin, OpenXR
• Rendering: Universal Render Pipeline
• Target Frame Rate: 90 FPS

Performance Optimization

• Texture Quality: Optimized for mobile VR
• Polygon Count: LOD system for complex models
• Audio Compression: Vorbis for speech audio
• Memory Management: Automatic garbage collection tuning

📖 Documentation

Core Components

• Setup Guide: Detailed installation and configuration
• API Documentation: Complete API reference
• VR Interaction Guide: VR-specific features
• AI Integration Guide: AI system details

Development Resources

• Contributing Guidelines: How to contribute to the project
• Code Style Guide: Coding standards and practices
• Performance Guide: Optimization best practices
• Troubleshooting: Common issues and solutions

🤝 Contributing

We welcome contributions from the medical and VR development communities!

Areas for Contribution

• Medical Content: Anatomical accuracy and educational scenarios
• Language Support: Additional language translations
• VR Optimization: Performance improvements and new VR features
• AI Enhancement: Better medical AI responses and context handling
• Accessibility: Features for users with different abilities

Development Setup

# Fork the repository
git clone https://github.com/your-username/HeartLab-VR-AI.git
cd HeartLab-VR-AI

# Create feature branch
git checkout -b feature/your-feature-name

# Make changes and commit
git commit -m "Add your feature"

# Push and create pull request
git push origin feature/your-feature-name

📊 Performance Targets

🛠️ Tech Stack

Core Technologies

• Unity 2022.3 LTS: Game engine and VR development
• C#: Primary programming language
• XR Interaction Toolkit: VR interaction framework
• Universal Render Pipeline: Optimized rendering

VR & Hardware

• Meta Quest SDK: Quest-specific optimizations
• OpenXR: Cross-platform VR compatibility
• Hand Tracking API: Natural hand interactions
• Oculus Audio SDK: Spatial audio processing

AI & Cloud Services

• Hugging Face Transformers: Medical AI models
• Azure Cognitive Services: Speech recognition and synthesis
• PubMedBERT: Medical domain language model
• ClinicalBERT: Clinical text understanding

Languages & Localization

• Turkish Medical Terminology: Comprehensive medical vocabulary
• Unicode Support: Multi-language text rendering
• Azure Speech: Turkish TTS and STT
• Cultural Adaptation: Turkish educational standards

📈 Roadmap

Version 1.1 (Q2 2024)

• Advanced haptic feedback patterns
• Multi-user collaborative sessions
• Extended anatomical structures (circulatory system)
• Assessment and testing modules

Version 1.2 (Q3 2024)

• AR mode for mobile devices
• Integration with medical curriculum standards
• Advanced AI tutoring system
• Real-time performance analytics

Version 2.0 (Q4 2024)

• Full body anatomy expansion
• Surgical simulation modules
• Medical professional certification
• Cloud-based progress tracking

📄 License

This project is licensed under the MIT License - see the LICENSE file for details.

🏥 Medical Disclaimer

This application is designed for educational purposes only and should not be used as a substitute for professional medical education or training. Always consult qualified medical professionals for clinical guidance.

📞 Support & Contact

Community

• GitHub Issues: Report bugs and request features
• Discussions: Community discussions and Q&A
• Wiki: Community-maintained documentation

Professional Support

• Technical Support: [email protected]
• Educational Partnerships: [email protected]
• Commercial Licensing: [email protected]

Social Media

• Twitter: @HeartLabVR
• LinkedIn: HeartLab VR
• YouTube: HeartLab VR Channel

---

Revolutionizing Medical Education Through Immersive VR and AI

Made with ❤️ by the HeartLab VR Team