Berta AI Scribe

Berta AI Scribe is an advanced medical documentation assistant designed to help healthcare providers efficiently create clinical notes from audio recordings of patient encounters. The system uses state-of-the-art AI transcription services and language models to transform medical conversations into well-structured clinical documentation.

Table of Contents

• Overview
• Features
• Architecture
• Technical Components
  • Transcription Services
  • Language Model Services
• Storage Configuration
• Database Configuration
• Local Development Setup
  • Quick Start with Docker Compose
  • Manual Setup
  • Prerequisites
  • Backend Environment Setup
  • Local Development Options
  • Option 1: OpenAI Setup (Easiest)
  • Option 2: Basic Local Setup (Offline)
  • Option 3: Local GPU Setup (VLLM)
  • Option 4: LM Studio Setup
  • Option 5: NVIDIA DGX Spark / GB10 Setup
  • Start the Backend
  • Frontend Setup
  • Verification
• AWS Deployment
  • Step 1: AWS Account Setup
  • Step 2: Domain Setup
  • Step 3: Create VPC Infrastructure (AWS Console)
  • Step 4: Deploy the Application
  • Step 5: Post-Deployment Configuration
• Available Services Reference
• Security
• Contributors
• License
  • Third-Party Licenses
  • Llama 3.3 License Notice
  • Attribution Notices
• Medical Disclaimer

Overview

Berta AI Scribe aims to reduce the documentation burden on healthcare providers by:

• Automatically transcribing patient encounters
• Generating structured clinical notes based on transcriptions
• Supporting various note templates for different clinical scenarios
• Providing a user-friendly interface for review and editing

Features

• Audio Recording & Transcription: Record patient encounters or upload existing audio files
• AI-Powered Note Generation: Generate comprehensive clinical notes from transcripts
• Multiple Note Templates: Support for various note formats (Full Visit, Narrative, Handover Notes, etc.)
• Custom Note Types: Create and save your own note templates
• Multi-Environment Support: Runs on AWS or local development environments
• Secure Authentication: Google OAuth or AWS Cognito context-based authentication

Architecture

The project consists of two main components:

1. Backend (web-api): A FastAPI-based service that handles:

   • Authentication
   • Audio processing and transcription
   • Note generation via LLMs
   • Database operations
   • File storage

2. Frontend (ai-scribe-app): A Next.js-based web application that provides:

   • User interface for recording or uploading audio
   • Note type selection and configuration
   • Review and editing of generated notes
   • User authentication flows

Basic Architecture

The system follows a modern web application architecture with several layers:

• Web Browser: The client interface accessed by users
• Next.js Frontend: Server-side rendered React application
• Python FastAPI Backend: Handles API requests and business logic
• Virtualization Layer: Contains:
  • vLLM Inference Engine: For AI model inference
  • Speech-to-text (ASR): For audio transcription
• Hardware Layer: GPU Cluster for high-performance computing
• Virtual Private Cloud: Secure network environment

Technical Components

Transcription Services

Berta Scribe supports four transcription services:

1. Parakeet MLX (Default): Local, fast transcription using Apple's MLX framework
2. OpenAI Whisper: State-of-the-art speech recognition via OpenAI API
3. WhisperX: Enhanced local Whisper with better accuracy and speed
4. AWS Transcribe: Cloud-based transcription with medical terminology support

The transcription service is configurable via the TRANSCRIPTION_SERVICE environment variable.

Language Model Services

The application supports six language model providers:

1. Ollama (Default): Local open-source models (any models available in ollama list)
2. OpenAI: GPT-4o via OpenAI API
3. AWS Bedrock: Meta Llama 3.3 70B, Llama 3.1 405B/70B, Claude 3.7 Sonnet
4. VLLM: Self-hosted inference server for large models
5. LM Studio: Local inference with user-friendly GUI and model management
6. LlamaCpp: High-performance llama.cpp server optimized for NVIDIA GPUs (recommended for DGX Spark)

The system will automatically use the best available model based on your configuration. For the local deployment we will be using gpt-4o via OpenAI API and for the AWS deployment we will be using Llama3.3 70b.

[!NOTE] The main note generation uses the model specified in your environment configuration. Additionally, the application provides custom settings where you can test different note instructions against various models:

• Local Development:
  • Ollama: All models from your ollama list appear as testing options in custom settings
  • LM Studio: Only currently loaded models in LM Studio appear as testing options (unlike Ollama which shows all downloaded models)
• AWS Deployment: A fixed set of Bedrock models (Meta Llama 3.3 70B, Llama 3.1 405B/70B, Claude 3.7 Sonnet) are available for testing custom note instructions

Storage Configuration

Berta Scribe supports two storage options:

1. Local Storage (Development):

   • Files stored in .data/recordings
   • Automatically configured for local development

2. S3 Storage (AWS Production):

   • Files stored in configured S3 bucket
   • Requires AWS credentials and bucket configuration

The storage provider is automatically selected based on environment variables.

Database Configuration

The application supports three database options:

1. SQLite (Development):

   • Automatically configured for local development
   • Database file stored in .data/database.db

2. Aurora PostgreSQL (AWS Production):

   • Configure using USE_AURORA=true
   • Requires Aurora writer endpoint and credentials

Local Development Setup

[!IMPORTANT] Before you begin: Local development requires Google OAuth credentials for authentication. You'll need to set up a Google Cloud project and create OAuth credentials before the application will work. See the Setting up Google OAuth section below.

Quick Start with Docker Compose

The fastest way to get started is using Docker Compose. This handles all dependencies (Python, Node.js, FFmpeg, audiowaveform) automatically.

Prerequisites:

• Docker Desktop installed and running
• Google OAuth credentials (see Setting up Google OAuth)
• OpenAI API key (for the default setup) or Ollama installed on your host machine

Steps:

1. Set up Google OAuth first (see instructions below)

2. Create environment file:

   cp .env.example .env
   

3. Edit .env with your credentials:

   ACCESS_TOKEN_SECRET=your_generated_secret    # Run: openssl rand -base64 32
   GOOGLE_CLIENT_ID=your_google_client_id
   GOOGLE_CLIENT_SECRET=your_google_client_secret
   OPENAI_API_KEY=your_openai_api_key           # If using OpenAI (default)
   

4. Start the application:

   docker compose up
   

   [!NOTE] First build takes 5-10 minutes (downloading ML dependencies and compiling audiowaveform). Subsequent starts are fast since Docker caches the build layers.

5. Access the app at http://localhost:4000

Stopping the application:

docker compose down

[!NOTE] Apple Silicon users: The containers run linux/amd64 via Rosetta emulation to match AWS production. This works correctly but may be slightly slower than native builds. Ensure "Use Rosetta for x86/amd64 emulation" is enabled in Docker Desktop → Settings → General.

---

Manual Setup

If you prefer not to use Docker, or need more control over the setup, follow the manual installation instructions below.

Prerequisites

• Python 3.11+ (managed with uv)
• uv (modern Python package and project manager)
• Node.js 18+ and npm (download from nodejs.org or use a version manager like nvm)
• FFmpeg (for audio processing)
• audiowaveform v1.10+ (for audio visualization)
• Google OAuth credentials (required - see setup guide below)

macOS users: Most dependencies can be installed via Homebrew. If you don't have Homebrew installed:

/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"

Intel Mac users: Note that the default transcription service (Parakeet MLX) only works on Apple Silicon (M1/M2/M3/M4). Intel Mac users should use OpenAI Whisper (Option 1) or WhisperX (Option 2 with WhisperX configuration). See the setup options below for specific instructions.

Installing uv (Python Package Manager)

macOS/Linux:

curl -LsSf https://astral.sh/uv/install.sh | sh

Windows:

powershell -ExecutionPolicy ByPass -c "irm https://astral.sh/uv/install.ps1 | iex"

Alternative (using pip):

pip install uv

Verify installation:

uv --version
# Should show uv version information

Installing FFmpeg

Ubuntu/Debian:

sudo apt update
sudo apt install ffmpeg

macOS:

# Using Homebrew
brew install ffmpeg

Windows:

# Using Chocolatey
choco install ffmpeg

Verify installation:

ffmpeg -version
# Should show FFmpeg version information

Installing audiowaveform

Ubuntu/Debian:

sudo add-apt-repository ppa:chris-needham/ppa
sudo apt-get update
sudo apt-get install audiowaveform

macOS:

brew install audiowaveform

Windows: Download from BBC audiowaveform releases or use WSL with Ubuntu instructions.

Verify installation:

audiowaveform --version
# Should show version 1.10 or higher

Setting up Google OAuth

For local development, you'll need Google OAuth credentials:

1. Go to the Google Cloud Console

2. Create a new project (or select existing one):

   • Click "Select a project" → "New Project"
   • Enter project name (e.g., "Berta Scribe Local")
   • Click "Create"

3. Navigate to "APIs & Services" → "Credentials"

4. Configure OAuth consent screen:

   • Click "OAuth consent screen"
   • Select "External" user type (for testing)
   • Fill in required fields:
     • App name: "Berta Scribe"
     • User support email: Your email
     • Developer contact information: Your email
   • Click "Save and Continue"
   • Skip scopes (click "Save and Continue")
   • Add test users if needed, or skip
   • Click "Back to Dashboard"

5. Create OAuth credentials:

   • Click "Create Credentials" → "OAuth client ID"
   • Choose "Web application"
   • Name: "Berta Scribe Local"
   • Authorized JavaScript origins:
     • http://localhost:4000
   • Authorized redirect URIs:
     • http://localhost:4000/login
   • Click "Create"

6. Note your credentials:

   • Copy the Client ID and Client Secret
   • Where to find them: After clicking "Create", a popup will show both credentials. If you miss this popup, go to "APIs & Services" → "Credentials" → click on your OAuth client name → the Client ID and Client Secret will be displayed on the details page
   • You'll need these for your environment files

[!IMPORTANT] The redirect URIs must match exactly. If you change the frontend port, update the redirect URIs accordingly.

Backend Environment Setup

Before configuring specific AI services, set up the Python backend environment:

1. Navigate to backend directory:

   cd web-api
   

2. Create Python virtual environment with uv:

   uv venv --python 3.11
   

3. Activate the virtual environment:

   # macOS/Linux
   source .venv/bin/activate
   
   # Windows
   .venv\Scripts\activate
   

4. Install dependencies:

   # BEFORE installing, check if you need to uncomment any dependencies:
   # - VLLM users: Uncomment vllm, torch, torchaudio lines
   # - Apple Silicon users: Uncomment mlx, parakeet-mlx lines
   # - Everyone else: No changes needed
   
   uv pip install -r requirements.txt
   

5. For WhisperX with GPU Support (Optional): If you plan to use WhisperX with an NVIDIA GPU (Option 2 or 3), upgrade PyTorch to CUDA version for faster transcription:

   # Only if you have NVIDIA GPU and want faster WhisperX transcription
   uv pip install torch==2.5.0 torchaudio==2.5.0 --index-url https://download.pytorch.org/whl/cu121 --reinstall --no-deps
   
   # Install cuDNN libraries (Ubuntu/Debian)
   wget https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2204/x86_64/cuda-keyring_1.1-1_all.deb
   sudo dpkg -i cuda-keyring_1.1-1_all.deb
   sudo apt-get update
   sudo apt-get install -y libcudnn8=8.9.7.29-1+cuda12.2
   
   # Set environment variable
   export LD_LIBRARY_PATH=/usr/local/cuda/lib64:$LD_LIBRARY_PATH
   

Then set WHISPERX_DEVICE=cuda in your .env file when configuring WhisperX.

Note: Skip this step if you're using OpenAI (Option 1), Apple Silicon with Parakeet MLX, or don't have an NVIDIA GPU.

[!NOTE] Keep this terminal open with the virtual environment activated for the remaining setup steps.

Local Development Options

All local setups use SQLite database, local file storage, and Google OAuth authentication. Choose based on your AI service preference:

[!IMPORTANT] If you're switching between different AI models or services, delete the .data folder in the web-api directory to clear any cached model data and ensure a clean start with your new configuration.

Common Environment Variables

Create the environment file:

1. Navigate to the web-api directory
2. Create a new file called .env (note the dot at the beginning)
3. Copy and paste the following settings into your new .env file:

# Core Settings
ENVIRONMENT=development
COOKIE_SECURE=false
LOGGING_LEVEL=DEBUG

# JWT Configuration
# ACCESS_TOKEN_SECRET: A random string used to sign JWT tokens for security
# Generate one with: openssl rand -base64 32
ACCESS_TOKEN_SECRET=your_secure_random_string_here
ACCESS_TOKEN_EXPIRE_MINUTES=1440

# Authentication (Google OAuth)
USE_COGNITO=false
USE_GOOGLE_AUTH=true
GOOGLE_CLIENT_ID=your_google_client_id_from_oauth_setup
GOOGLE_CLIENT_SECRET=your_google_client_secret_from_oauth_setup
GOOGLE_REDIRECT_URI=http://localhost:4000/login

# Database (Local SQLite)
USE_AURORA=false

Then, add the AI service-specific variables based on your chosen option below:

Option 1: OpenAI Setup (Easiest)

Best for: Quick start, highest quality AI models, minimal setup Uses: OpenAI Whisper transcription + GPT-4o models

Requirements:

• OpenAI API key (get one at platform.openai.com)
• Google OAuth credentials
• No local AI software needed!

Setup Steps:

1. Get OpenAI API Key:

   • Sign up/Login: Go to platform.openai.com and create an account or log in
   • Navigate to API Keys: Go to platform.openai.com/api-keys
   • Create new key: Click "Create new secret key"
   • Name your key: Give it a descriptive name like "Berta Scribe Local"
   • Set permissions: Choose "All" or ensure it has access to the models you need
   • Copy the key: After creation, copy the API key immediately (it starts with sk-...)
   • Save securely: Store it in a safe place - you won't be able to see it again
   • Add billing: Make sure you have billing set up at platform.openai.com/settings/organization/billing

2. Add OpenAI settings to your .env file:

   • Open the web-api/.env file you created earlier
   • Append these lines at the bottom (below the common settings):

   # AI Services (OpenAI)
   TRANSCRIPTION_SERVICE=OpenAI Whisper
   GENERATIVE_AI_SERVICE=OpenAI
   DEFAULT_NOTE_GENERATION_MODEL=gpt-4o
   LABEL_MODEL=gpt-4o
   
   # OpenAI API Key (replace with your actual API key from step 1)
   OPENAI_API_KEY=your_openai_api_key_here
   

3. That's it! No additional software to install or configure.

[!NOTE] Costs: OpenAI charges approximately $0.36 per hour of audio transcribed. GPT-4o usage is additional but typically minimal for note generation.

Option 2: Basic Local Setup (Offline)

Best for: First-time users, completely offline setup Uses: Parakeet MLX transcription + Ollama models

[!WARNING] Parakeet MLX requires Apple Silicon (M1/M2/M3 Macs). If you're on Intel Mac, Linux, or Windows, you must change the transcription service in step 5 below.

Requirements:

• No external API keys needed
• Works completely offline
• Google OAuth credentials

Setup Steps:

1. Install Ollama:

   # macOS
   brew install ollama
   
   # Linux
   curl -fsSL https://ollama.ai/install.sh | sh
   
   # Windows - Download from https://ollama.ai/download
   

2. Start Ollama service:

   # Start Ollama service (required for the application to work)
   ollama serve
   
   # The service will run on http://localhost:11434
   # Keep this terminal open or run as a background service
   

3. Pull Ollama models (in a new terminal):

   ollama pull llama3.1:8b
   # Optional: For better quality (requires more RAM)
   # ollama pull llama3.3:70b
   

4. Verify Ollama is working:

   ollama list
   # Should show your downloaded models
   
   curl http://localhost:11434/api/tags
   # Should return JSON with available models
   

[!IMPORTANT] Apple Silicon users: Make sure you uncommented the mlx and parakeet-mlx lines in requirements.txt before installing dependencies (as mentioned in Backend Environment Setup step 4).

[!NOTE] Any models you have already downloaded with Ollama (visible in ollama list) will automatically appear as options in the application's custom settings, allowing you to test different note instructions with various models.

5. Append these lines to your web-api/.env file (below the common settings): For Apple Silicon Mac:

   # AI Services (Ollama)
   TRANSCRIPTION_SERVICE=Parakeet MLX
   GENERATIVE_AI_SERVICE=Ollama
   DEFAULT_NOTE_GENERATION_MODEL=llama3.1:8b
   LABEL_MODEL=llama3.1:8b
   

   For Intel/Linux/Windows with NVIDIA GPU:

   # AI Services (Ollama + WhisperX GPU)
   TRANSCRIPTION_SERVICE=WhisperX
   WHISPERX_DEVICE=cuda  # Fast GPU transcription
   GENERATIVE_AI_SERVICE=Ollama
   DEFAULT_NOTE_GENERATION_MODEL=llama3.1:8b
   LABEL_MODEL=llama3.1:8b
   

   For Intel/Linux/Windows CPU-only:

   # AI Services (Ollama + WhisperX CPU)
   TRANSCRIPTION_SERVICE=WhisperX
   WHISPERX_DEVICE=cpu  # Warning: Slow transcription (consider Option 1 instead)
   GENERATIVE_AI_SERVICE=Ollama
   DEFAULT_NOTE_GENERATION_MODEL=llama3.1:8b
   LABEL_MODEL=llama3.1:8b
   

[!WARNING] Performance Notes:

• Parakeet MLX (Apple Silicon): Fast, efficient transcription
• WhisperX GPU (NVIDIA): Fast transcription, comparable to Parakeet
• WhisperX CPU: Very slow (5-20x slower than real-time). Consider using Option 1 (OpenAI) for better performance if you don't have Apple Silicon or NVIDIA GPU.

[!NOTE] Any models you have already downloaded with Ollama (visible in ollama list) will automatically appear as options in the application's custom settings, allowing you to test different note instructions with various models.

Option 3: Local GPU Setup (VLLM)

Best for: Users with powerful GPUs, maximum performance and privacy Uses: VLLM inference + WhisperX transcription

Requirements:

• NVIDIA GPU with 8GB+ VRAM
• CUDA toolkit installed
• Google OAuth credentials
• Hugging Face token

Setup Steps:

1. Install CUDA toolkit (if not installed):

   # Check if CUDA is installed
   nvidia-smi
   
   # Ubuntu/Debian installation
   wget https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2204/x86_64/cuda-keyring_1.1-1_all.deb
   sudo dpkg -i cuda-keyring_1.1-1_all.deb
   sudo apt-get update
   sudo apt-get install cuda-toolkit-12-4
   

2. Modify requirements.txt and install VLLM (in the Python virtual environment):

   cd web-api
   # First, uncomment these lines in requirements.txt:
   # vllm>=0.3.0
   # torch>=2.5.0,<3.0.0
   # torchaudio>=2.5.0,<3.0.0
   # nvidia-cudnn-cu12>=9.0.0  # Optional but recommended
   
   # Then install:
   uv pip install -r requirements.txt
   
   

3. Get Hugging Face token:

   • Go to huggingface.co/settings/tokens
   • Create a new token with "Read" permissions
   • Accept the Llama model license at huggingface.co/meta-llama

4. Append these lines to your web-api/.env file (below the common settings):

   # AI Services (VLLM)
   TRANSCRIPTION_SERVICE=WhisperX
   GENERATIVE_AI_SERVICE=VLLM
   
   # VLLM Configuration
   VLLM_SERVER_NAME=localhost
   VLLM_SERVER_PORT=8080
   
   VLLM_MODEL_NAME=meta-llama/Meta-Llama-3.1-70B-Instruct
   DEFAULT_NOTE_GENERATION_MODEL=meta-llama/Meta-Llama-3.1-70B-Instruct
   LABEL_MODEL=meta-llama/Meta-Llama-3.1-70B-Instruct
   
   # Hugging Face token (required for model downloads)
   HUGGINGFACE_TOKEN=your_huggingface_token
   
   # WhisperX device (if using WhisperX): cuda, cpu, or cuda:0
   WHISPERX_DEVICE=cuda
   

[!IMPORTANT] All three model variables must have the same value:

• VLLM_MODEL_NAME - Specifies which model to download from Hugging Face
• DEFAULT_NOTE_GENERATION_MODEL - Model used for generating clinical notes
• LABEL_MODEL - Model used for note labeling and classification These must match exactly for VLLM to work properly.

5. Start VLLM server:

   # Start VLLM server in separate terminal
   python -m vllm.entrypoints.openai.api_server \
     --model meta-llama/Meta-Llama-3.1-70B-Instruct \
     --host localhost \
     --port 8080 \
     --gpu-memory-utilization 0.95
   

Option 4: LM Studio Setup

Best for: Users who want a GUI for model management and high-quality local inference Uses: Parakeet MLX transcription + LM Studio models

Requirements:

• LM Studio installed
• Google OAuth credentials
• No external API keys needed

[!IMPORTANT] Apple Silicon users: Make sure you uncommented the mlx and parakeet-mlx lines in requirements.txt before installing dependencies. Non-Apple Silicon users: Change TRANSCRIPTION_SERVICE to WhisperX or OpenAI Whisper in step 5, as Parakeet MLX only works on Apple Silicon.

Setup Steps:

1. Install LM Studio:

   • Download from lmstudio.ai
   • Install and launch LM Studio

2. Download models in LM Studio:

   • Open LM Studio
   • Go to "Search" tab
   • Download models like:
     • llama-3.1-8b-instruct (faster, 8GB RAM)
     • llama-3.3-70b-instruct (higher quality, 64GB+ RAM)
     • mistral-7b-instruct-v0.3 (good balance)

3. Load a model:

   • Go to "Chat" tab in LM Studio
   • Click "Select a model to load"
   • Choose your preferred model and click "Load Model"
   • Wait for the model to load completely

4. Start LM Studio server:

   • In LM Studio, go to "Local Server" tab
   • Click "Start Server"
   • Note the server URL (usually http://localhost:1234)

[!NOTE] Unlike Ollama which shows all downloaded models in custom settings, LM Studio only shows the currently loaded model as an option for testing different note instructions. You must load the desired model in LM Studio's interface before it becomes available in the application.

5. Append these lines to your web-api/.env file (below the common settings):

   # AI Services (LM Studio)
   TRANSCRIPTION_SERVICE=Parakeet MLX
   GENERATIVE_AI_SERVICE=LM Studio
   
   # Model Selection (use the name of the loaded model in LM Studio)
   DEFAULT_NOTE_GENERATION_MODEL=llama-3.1-8b-instruct
   LABEL_MODEL=llama-3.1-8b-instruct
   

[!IMPORTANT] Make sure LM Studio server is running and a model is loaded before starting the backend. The model name in your environment file should match the loaded model in LM Studio.

Option 5: NVIDIA DGX Spark / GB10 Setup (ARM64 + CUDA 13)

Best for: NVIDIA DGX Spark workstations with GB10 chip (Project DIGITS) Uses: WhisperX GPU transcription + Ollama with MedGemma or other medical LLMs

[!IMPORTANT] The NVIDIA GB10 uses ARM64 architecture with CUDA 13.0, which requires building some dependencies from source due to limited pre-built wheel availability.

Requirements:

• NVIDIA DGX Spark with GB10 chip
• Ubuntu 24.04 LTS (typical DGX Spark OS)
• CUDA 13.0 toolkit (pre-installed on DGX Spark)
• Google OAuth credentials

Setup Steps:

1. Install system dependencies:

   sudo apt update
   sudo apt install -y ffmpeg libboost-all-dev libmad0-dev libid3tag0-dev \
       libsndfile1-dev libgd-dev cmake git build-essential
   

2. Build audiowaveform from source (no ARM64 binaries available):

   cd /tmp
   git clone https://github.com/bbc/audiowaveform.git
   cd audiowaveform
   mkdir build && cd build
   cmake .. -DENABLE_TESTS=OFF
   make -j$(nproc)
   sudo make install
   

3. Set up Python environment:

   cd web-api
   uv venv --python 3.11
   source .venv/bin/activate
   uv pip install -r requirements.txt
   

4. Install PyTorch with CUDA 13 support:

   uv pip install torch torchaudio --index-url https://download.pytorch.org/whl/cu130
   

5. Build CTranslate2 from source with CUDA 13 (no pre-built ARM64 CUDA wheels):

   # Install pybind11
   uv pip install pybind11
   
   # Clone and build CTranslate2
   cd /tmp
   git clone --recursive https://github.com/OpenNMT/CTranslate2.git
   cd CTranslate2
   mkdir build && cd build
   cmake .. -DWITH_CUDA=ON -DWITH_CUDNN=OFF -DWITH_MKL=OFF -DWITH_OPENBLAS=OFF \
       -DCMAKE_BUILD_TYPE=Release -DOPENMP_RUNTIME=NONE
   make -j$(nproc)
   cmake --install . --prefix /tmp/ctranslate2_install
   
   # Install Python bindings
   cd /tmp/CTranslate2/python
   CTranslate2_ROOT=/tmp/ctranslate2_install \
   CMAKE_PREFIX_PATH=/tmp/ctranslate2_install \
   CPLUS_INCLUDE_PATH=/tmp/ctranslate2_install/include \
   LIBRARY_PATH=/tmp/ctranslate2_install/lib \
   uv pip install . --no-build-isolation
   

6. Install and configure Ollama:

   curl -fsSL https://ollama.ai/install.sh | sh
   ollama serve &
   
   # Pull a medical LLM (example: MedGemma)
   ollama pull MedAIBase/MedGemma1.5:4b
   

7. Configure environment - Append to your web-api/.env file:

   # AI Services (WhisperX GPU + Ollama)
   TRANSCRIPTION_SERVICE=WhisperX
   WHISPERX_DEVICE=cuda
   GENERATIVE_AI_SERVICE=Ollama
   
   # Model names must include the tag (e.g., :4b)
   DEFAULT_NOTE_GENERATION_MODEL=MedAIBase/MedGemma1.5:4b
   LABEL_MODEL=MedAIBase/MedGemma1.5:4b
   

8. Start the backend (requires environment variables):

   cd web-api
   source .venv/bin/activate
   LD_LIBRARY_PATH=/tmp/ctranslate2_install/lib TORCH_FORCE_NO_WEIGHTS_ONLY_LOAD=1 \
       uvicorn app.main:app --reload --host 0.0.0.0 --port 8000
   

[!TIP] Create a startup script start-backend.sh for convenience:

#!/bin/bash
export LD_LIBRARY_PATH=/tmp/ctranslate2_install/lib:$LD_LIBRARY_PATH
export TORCH_FORCE_NO_WEIGHTS_ONLY_LOAD=1
cd ~/projects/berta-ai-scribe/web-api
source .venv/bin/activate
uvicorn app.main:app --reload --host 0.0.0.0 --port 8000

Alternative: Using LlamaCpp (Faster Inference)

For better performance on DGX Spark, you can use llama.cpp instead of Ollama. llama.cpp is ~35% faster and supports Blackwell-native optimizations.

1. Build llama.cpp with CUDA 13 and Blackwell support:

   cd ~
   git clone https://github.com/ggerganov/llama.cpp.git
   cd llama.cpp
   mkdir build-gpu && cd build-gpu
   cmake .. -DCMAKE_BUILD_TYPE=Release -DGGML_CUDA=ON -DGGML_CUDA_F16=ON -DCMAKE_CUDA_ARCHITECTURES=121
   make -j$(nproc)
   

2. Download a GGUF model (example: Llama 3.3 70B Q4):

   mkdir -p ~/models
   cd ~/models
   # Download from Hugging Face (one-time, runs 100% locally after)
   wget https://huggingface.co/bartowski/Llama-3.3-70B-Instruct-GGUF/resolve/main/Llama-3.3-70B-Instruct-Q4_K_M.gguf
   

3. Start llama-server:

   cd ~/llama.cpp/build-gpu
   LD_LIBRARY_PATH=./bin:$LD_LIBRARY_PATH ./bin/llama-server \
     -m ~/models/Llama-3.3-70B-Instruct-Q4_K_M.gguf \
     -ngl 99 -c 4096 --host 0.0.0.0 --port 8080
   

4. Configure environment - Use these settings in your web-api/.env file:

   # AI Services (WhisperX GPU + LlamaCpp)
   TRANSCRIPTION_SERVICE=WhisperX
   WHISPERX_DEVICE=cuda
   GENERATIVE_AI_SERVICE=LlamaCpp
   # LLAMA_CPP_SERVER_URL=http://localhost:8080  # Optional, defaults to localhost:8080
   
   # Model name must match the loaded GGUF file
   DEFAULT_NOTE_GENERATION_MODEL=Llama-3.3-70B-Instruct-Q4_K_M.gguf
   LABEL_MODEL=Llama-3.3-70B-Instruct-Q4_K_M.gguf
   

5. Start the backend (in a separate terminal):

   cd web-api
   source .venv/bin/activate
   LD_LIBRARY_PATH=~/ctranslate2_install/lib:$LD_LIBRARY_PATH TORCH_FORCE_NO_WEIGHTS_ONLY_LOAD=1 \
       uvicorn app.main:app --reload --host 0.0.0.0 --port 8000
   

[!TIP] Startup Order: Start llama-server first (wait ~60 seconds for model to load), then start the backend.

[!WARNING] Known Warnings (can be safely ignored):

• PyTorch may warn about CUDA capability 12.1 vs supported 12.0 - this generally works fine
• pyannote.audio version mismatch warnings - models still function correctly

Alternative: Using vLLM Docker (NVIDIA Optimized - Best for Scaling)

For production deployments and better scaling, use NVIDIA's optimized vLLM Docker container. vLLM offers continuous batching, PagedAttention for efficient memory use, and tensor parallelism for multi-GPU setups.

[!IMPORTANT] GPU Sharing: Docker containers take exclusive GPU access. Start the backend (WhisperX) BEFORE launching the vLLM Docker container to allow both to coexist on unified memory.

1. Pull the NVIDIA-optimized vLLM container:

   docker pull nvcr.io/nvidia/vllm:26.01-py3
   

2. Choose your model based on available memory:

   Model	Memory Required	Command
   Llama 3.1 8B (recommended for GPU sharing)	~16GB	See below
   Llama 3.3 70B NVFP4 (Blackwell-optimized 4-bit)	~40GB	See below

3. Start vLLM Docker (choose one):

   For Llama 3.1 8B (leaves ~50GB for WhisperX):

   docker run --gpus all -p 8080:8080 \
     -e HUGGING_FACE_HUB_TOKEN=your_hf_token \
     nvcr.io/nvidia/vllm:26.01-py3 \
     --model meta-llama/Llama-3.1-8B-Instruct \
     --tensor-parallel-size 1 \
     --gpu-memory-utilization 0.65 \
     --port 8080
   

   For Llama 3.3 70B with NVFP4 (Blackwell 4-bit quantization, ~3.3x memory reduction):

   docker run --gpus all -p 8080:8080 \
     -e HUGGING_FACE_HUB_TOKEN=your_hf_token \
     nvcr.io/nvidia/vllm:26.01-py3 \
     --model neuralmagic/Meta-Llama-3.3-70B-Instruct-nvfp4 \
     --tensor-parallel-size 1 \
     --gpu-memory-utilization 0.65 \
     --port 8080
   

4. Configure environment - Use these settings in your web-api/.env file:

   # AI Services (WhisperX GPU + vLLM Docker)
   TRANSCRIPTION_SERVICE=WhisperX
   WHISPERX_DEVICE=cuda
   GENERATIVE_AI_SERVICE=VLLM
   
   # vLLM Configuration
   VLLM_SERVER_NAME=localhost
   VLLM_SERVER_PORT=8080
   
   # Model name must match exactly what vLLM loads
   VLLM_MODEL_NAME=meta-llama/Llama-3.1-8B-Instruct
   DEFAULT_NOTE_GENERATION_MODEL=meta-llama/Llama-3.1-8B-Instruct
   LABEL_MODEL=meta-llama/Llama-3.1-8B-Instruct
   

5. Startup order (critical for GPU sharing):

   # Terminal 1: Start backend FIRST (initializes WhisperX on GPU)
   cd web-api && source .venv/bin/activate
   LD_LIBRARY_PATH=~/ctranslate2_install/lib TORCH_FORCE_NO_WEIGHTS_ONLY_LOAD=1 \
       uvicorn app.main:app --reload --host 0.0.0.0 --port 8000
   
   # Terminal 2: Start vLLM Docker AFTER backend is running
   docker run --gpus all -p 8080:8080 ...
   

[!TIP] Scaling vLLM:

• Multi-GPU: Use --tensor-parallel-size 2 (or higher) to split model across GPUs
• Multiple instances: Run several vLLM containers behind a load balancer
• Continuous batching: vLLM automatically batches concurrent requests for 2-4x throughput
• Check models: curl http://localhost:8080/v1/models to verify loaded model name

[!NOTE] Switching models: If you previously used a different model, delete the .data folder to reset the database:

rm -rf .data/

The database will be recreated with the correct model names from your .env file on next startup.```

Start the Backend

For ALL Users

• Add Google OAuth Client ID and Secret to web-api/.env (backend) file
• If switching AI services:
  • Delete the .data folder in web-api directory
  • Clear browser cache and storage for localhost:4000 (F12 → Application tab → Clear storage)

For Ollama Users (Option 2)

Start Ollama service FIRST:

ollama serve
# Keep this terminal open, then start backend in new terminal

For LM Studio Users (Option 4)

Before starting backend:

1. Open LM Studio
2. Load your model (from Chat tab)
3. Start server (from Local Server tab, use default settings)
4. Model name in .env must match exactly what's loaded in LM Studio

Startup Order

1. Start AI service (Ollama/LM Studio)
2. Start backend (uvicorn app.main:app --reload --port 8000)
3. Start frontend (npm run dev)

After completing your chosen AI service setup above:

1. Ensure your virtual environment is activated:

   # If not already activated from the Backend Environment Setup
   cd web-api
   source .venv/bin/activate  # macOS/Linux
   # or .venv\Scripts\activate  # Windows
   

2. Start the backend server:

   uvicorn app.main:app --reload --port 8000
   

Frontend Setup

1. Create frontend environment file:

   • Navigate to the ai-scribe-app directory
   • Create a new file called .env (note the dot at the beginning)
   • Copy and paste the following settings into your new .env file:

   # Backend API URL
   NEXT_PUBLIC_BACKEND_URL=http://localhost:8000
   
   # Authentication Configuration
   NEXT_PUBLIC_USE_COGNITO=false
   NEXT_PUBLIC_USE_GOOGLE_AUTH=true
   
   # Google OAuth Configuration (use same Client ID from backend setup)
   GOOGLE_CLIENT_ID=your_google_client_id_from_step_above
   GOOGLE_REDIRECT_URI=http://localhost:4000/login
   

2. Navigate to frontend directory:

   cd ai-scribe-app
   

3. Install dependencies:

   npm install
   

4. Start the frontend development server:

   npm run dev
   

   The frontend will be available at http://localhost:4000

[!NOTE] The GOOGLE_CLIENT_ID should be the same in both frontend and backend environment files.

Verification

1. Navigate to http://localhost:4000
2. Click the login button
3. Complete authentication flow
4. Test audio recording or file upload
5. Verify note generation works

Troubleshooting

Common Issues

Ollama Connection Issues:

• Ensure ollama serve is running in a separate terminal
• Check that Ollama is accessible at http://localhost:11434
• Verify models are downloaded with ollama list

Python Environment Issues:

• Make sure you're using Python 3.11+ with python --version
• Activate the virtual environment before installing dependencies
• If uv installation fails, try the pip alternative: pip install uv

Authentication Issues:

• Verify Google OAuth redirect URIs match exactly
• Check that both frontend and backend have the same Google Client ID
• Ensure the frontend is running on the port specified in OAuth settings (default: 4000)

Google OAuth Errors:

• "redirect_uri_mismatch": The redirect URI in your OAuth credentials doesn't match your app. Verify http://localhost:4000/login is in your authorized redirect URIs
• "access_denied": You may need to add your email as a test user in the Google Cloud Console (OAuth consent screen → Test users)
• "invalid_client": Double-check your Client ID and Client Secret are correctly copied to both .env files

Port Conflicts:

• Backend port 8000 in use: Check for other processes with lsof -i :8000 (macOS/Linux) or netstat -ano | findstr :8000 (Windows). Kill the process or use a different port with --port 8001
• Frontend port 4000 in use: Check with lsof -i :4000. If you change the port, remember to update your Google OAuth redirect URIs accordingly
• Ollama port 11434 in use: Another Ollama instance may be running. Check with ps aux | grep ollama

Service Startup Order:

• Services must start in the correct order: AI service (Ollama/LM Studio) → Backend → Frontend
• If the backend starts before Ollama, it may fail to connect. Restart the backend after Ollama is running
• LM Studio users: Ensure a model is loaded AND the server is started before launching the backend

Transcription Issues (Intel Mac / Windows / Linux):

• Parakeet MLX only works on Apple Silicon. If you see MLX-related errors on Intel/AMD systems, switch to WhisperX or OpenAI Whisper
• WhisperX first run downloads models (~1-2GB) which may take time
• For WhisperX GPU errors, ensure CUDA is properly installed with nvidia-smi

AWS Deployment

Architecture

Step 1: AWS Account Setup

1. Create AWS Account: If you don't have one, sign up at aws.amazon.com

2. Log into AWS Console: After creating your account, log into the AWS Management Console

3. Enable Bedrock Model Access:

   • Go to AWS Bedrock Console: In the AWS search bar at the top, type "Bedrock" and click on "Amazon Bedrock"
   • Navigate to "Model access" in the left sidebar
   • Request access to:
     • Meta Llama 3.3 70B Instruct (us.meta.llama3-3-70b-instruct-v1:0)
     • Meta Llama 3.1 405B Instruct (meta.llama3-1-405b-instruct-v1:0)
     • Meta Llama 3.1 70B Instruct (meta.llama3-1-70b-instruct-v1:0)
     • Anthropic Claude 3.7 Sonnet (anthropic.claude-3-7-sonnet-20250219-v1:0)

Step 2: Domain Setup

1. Register a Domain:

   Option 1: Register through Route53 Console (Recommended):

   • Go to Route53 Console
   • Click "Register Domain"
   • Search for your desired domain name
   • Follow the registration process (requires contact information and payment)
   • Domain registration can take up to 48 hours to complete

   Option 2: Use existing domain with Route53:

   # If you have a domain registered elsewhere, create a hosted zone
   aws route53 create-hosted-zone \
     --name yourdomain.com \
     --caller-reference $(date +%s) \
     --hosted-zone-config Comment="Berta Scribe hosted zone"
   
   # Note: You'll need to update your domain's nameservers to point to Route53
   

2. Find your Hosted Zone ID:

   Method 1 (AWS Console - Recommended):

   • Go to Route53 Console
   • Click "Hosted zones" in the left sidebar
   • Find your domain in the list
   • Copy the Hosted zone ID (looks like Z1D633PJN98FT9) - you'll need this for deployment

   Method 2 (AWS CLI):

   aws route53 list-hosted-zones --query "HostedZones[?Name=='yourdomain.com.'].Id" --output text
   

Step 3: Create VPC Infrastructure (AWS Console)

Option A: Use Existing VPC (If You Have One)

If you already have a VPC set up:

1. Note Your VPC Details:

   • VPC ID: Copy your VPC ID from the VPC console
   • Public Subnets: Copy the subnet IDs from your public subnets
   • Private Subnets: Copy the subnet IDs from your private subnets

2. Verify your subnets (run this command to check):

   aws ec2 describe-subnets --filters "Name=vpc-id,Values=<YOUR_VPC_ID>" --region us-west-2 \
     --query 'Subnets[*].{ID:SubnetId,AZ:AvailabilityZone,CIDR:CidrBlock,Public:MapPublicIpOnLaunch}' --output table
   

3. Verify you have a NAT Gateway:

   aws ec2 describe-nat-gateways --filter "Name=vpc-id,Values=<YOUR_VPC_ID>" --region us-west-2 \
     --query 'NatGateways[*].{ID:NatGatewayId,State:State,SubnetId:SubnetId}' --output table
   

4. Skip to Step 3b to add security hardening, then proceed to Step 4

Option B: Create New VPC (Recommended for New Users)

Use AWS VPC Wizard

1. Go to VPC Console:

   • Open VPC Console
   • Click "Create VPC"

2. VPC Settings - Choose "VPC and more":

   Setting	Value
   Resources to create	VPC and more
   Name tag auto-generation	berta
   IPv4 CIDR block	10.0.0.0/16
   IPv6 CIDR block	No IPv6 CIDR block
   Tenancy	Default
   Number of AZs	2
   Number of public subnets	2
   Number of private subnets	2
   NAT gateways	In 1 AZ
   VPC endpoints	S3 Gateway
   DNS hostnames	Enabled
   DNS resolution	Enabled

3. Review the Preview - You should see:

   • 4 subnets (2 public, 2 private)
   • 3 route tables
   • 3 network connections (IGW, NAT Gateway, VPC-S3)

4. Click "Create VPC" - AWS creates everything automatically!

5. Note Your Resource IDs (you'll need these for deployment):

   Resource	Where to Find
   VPC ID	VPC Details tab
   Public Subnets	Subnets with "public" in name (typically 10.0.0.0/20, 10.0.16.0/20)
   Private Subnets	Subnets with "private" in name (typically 10.0.128.0/20, 10.0.144.0/20)

Step 3b: Security Hardening (Recommended)

After creating your VPC, add these security measures:

Enable VPC Flow Logs (for monitoring)

VPC Flow Logs help you monitor network traffic and detect suspicious activity:

# Create CloudWatch log group
aws logs create-log-group --log-group-name /vpc/berta-flow-logs --region us-west-2

# Create IAM role for flow logs
echo '{"Version":"2012-10-17","Statement":[{"Effect":"Allow","Principal":{"Service":"vpc-flow-logs.amazonaws.com"},"Action":"sts:AssumeRole"}]}' > /tmp/trust-policy.json

aws iam create-role --role-name VPCFlowLogsRole --assume-role-policy-document file:///tmp/trust-policy.json

echo '{"Version":"2012-10-17","Statement":[{"Effect":"Allow","Action":["logs:CreateLogStream","logs:PutLogEvents","logs:DescribeLogGroups","logs:DescribeLogStreams"],"Resource":"*"}]}' > /tmp/flow-logs-policy.json

aws iam put-role-policy --role-name VPCFlowLogsRole --policy-name FlowLogsPolicy --policy-document file:///tmp/flow-logs-policy.json

# Enable flow logs on your VPC (replace <YOUR_VPC_ID> and <YOUR_ACCOUNT_ID>)
aws ec2 create-flow-logs --resource-type VPC --resource-ids <YOUR_VPC_ID> --traffic-type ALL \
  --log-destination-type cloud-watch-logs --log-group-name /vpc/berta-flow-logs \
  --deliver-logs-permission-arn arn:aws:iam::<YOUR_ACCOUNT_ID>:role/VPCFlowLogsRole --region us-west-2

Add Network ACL Rules (defense in depth)

Add DENY rules to block common malicious ports on your private subnet Network ACL:

1. Go to VPC Console → Network ACLs

2. Select the NACL associated with your private subnets

3. Edit Outbound Rules - Add these DENY rules (lower rule numbers = higher priority):

   Rule #	Type	Port	Destination	Action
   50	TCP	23	0.0.0.0/0	DENY
   51	TCP	445	0.0.0.0/0	DENY
   52	TCP	2323	0.0.0.0/0	DENY
   53	TCP	3389	0.0.0.0/0	DENY
   54	TCP	3306	0.0.0.0/0	DENY
   100	ALL	ALL	0.0.0.0/0	ALLOW

[!NOTE] These rules block common ports used by malware for scanning (Telnet, SMB, RDP, MySQL). The CloudFormation template already includes restrictive security group rules, but Network ACLs provide an additional layer of protection.

Step 4: Deploy the Application

Now you'll deploy Berta Scribe application using AWS CloudFormation:

Option A: One-click Deployment (Recommended)

1. Click the deployment button:

   Service	Button
   AWS

2. You'll be taken to the AWS CloudFormation console where you'll see a form to fill out

Option B: Custom Deployment

If you need to modify the CloudFormation template (e.g., change instance sizes, add custom configurations), you can use the template.yaml file included in this repository. Download the template, make your modifications, and deploy it manually through the AWS CloudFormation console or AWS CLI instead of using the one-click deployment above.

[!IMPORTANT] If you modify the template.yaml file and deploy it manually, you cannot use the one-click deployment button. You must deploy your custom template through the AWS CloudFormation console or CLI.

Fill in the required parameters:

3. Deploy the stack:

   • After filling in all parameters, scroll down
   • Check the box "I acknowledge that AWS CloudFormation might create IAM resources"
   • Click "Create stack"
   • The deployment will take 15-20 minutes

4. Monitor the deployment:

   • You'll see the CloudFormation stack creation in progress
   • Watch the "Events" tab to see resources being created
   • Wait for the stack status to show "CREATE_COMPLETE"
   • If deployment fails, check the "Events" tab for error details

5. Get your application URLs:

   • Once deployment is complete, go to the "Outputs" tab
   • Note down the FrontendURL - this is where you'll access the application

Step 5: Post-Deployment Configuration

Test the application:

• Navigate to the Frontend URL
• Complete Cognito authentication
• Test audio recording and note generation

Docker Images: The CloudFormation template uses pre-built Docker images hosted on AWS Public ECR:

• Frontend: public.ecr.aws/s9f8j1d3/berta-frontend:latest
• Backend: public.ecr.aws/s9f8j1d3/berta-backend:latest

These images are automatically pulled during deployment and contain the latest stable versions of the application components.

Updates: When new releases are available, we update the images at the same URLs. To get the latest version, simply restart your ECS services:

aws ecs update-service --cluster berta-cluster-production --service berta-frontend-production --force-new-deployment
aws ecs update-service --cluster berta-cluster-production --service berta-backend-production --force-new-deployment

Platform Support

Berta Scribe currently supports AWS for cloud production deployments. Support for Azure, GCP, and Databricks is under consideration based on community interest. If you need support for a specific platform, please open an issue on GitHub.

Available Services Reference

You can view all available services and models by running:

cd web-api
python -m app.cli.list_services

This will show:

• Transcription Services: Parakeet MLX, OpenAI Whisper, WhisperX, AWS Transcribe
• AI Services: Ollama (with your installed models), OpenAI, AWS Bedrock, VLLM, LM Studio
• Available Models:
  • Ollama: All models from your ollama list output
  • LM Studio: Only currently loaded models in LM Studio (must be loaded in LM Studio interface first)
  • AWS Bedrock: us.meta.llama3-3-70b-instruct-v1:0, meta.llama3-1-405b-instruct-v1:0, meta.llama3-1-70b-instruct-v1:0, anthropic.claude-3-7-sonnet-20250219-v1:0
  • OpenAI: gpt-4o, gpt-3.5-turbo
  • VLLM: Custom models you've configured

Security

Berta Scribe implements robust security measures:

• Secure authentication through Cognito or Google OAuth
• HTTPS for all external communication
• JWT tokens for API security
• Secure cookie handling
• Database encryption at rest
• S3 bucket encryption and private access
• Proper IAM roles and security groups in AWS

Contributors

• Samridhi Vaid MSc
• Michael Weldon MD MSc
• Jesse Dunn
• Kevin Lonergan
• Henry Li
• Jeffrey Franc
• Mohamed Abdala
• Daniel C. Baumgart
• Jake Hayward MD MPH
• Ross Mitchell PhD

License

Apache License

Third-Party Licenses

This project uses third-party libraries and models, including:

• WhisperX: Licensed under the BSD 2-Clause License.
• Meta Llama 3.3: Licensed under the Meta Llama 3.3 Community License Agreement.
• Ollama: Licensed under the MIT License.
• vLLM: Licensed under the Apache License 2.0.
• Parakeet Models: Licensed under the Creative Commons Attribution 4.0 International License (CC-BY-4.0).

For the full text of these licenses, please see the THIRD_PARTY_LICENSES file in this repository.

Llama 3.3 License Notice

This project uses Meta Llama 3.3. As per the Llama 3.3 license requirements:

• This project is "Built with Meta Llama 3.3".
• Any AI models created, trained, or fine-tuned using Llama 3.3 as part of this project will include "Llama 3.3" at the beginning of the model name.
• Use of Llama 3.3 in this project complies with the Meta Llama 3.3 Acceptable Use Policy.

For the complete Meta Llama 3.3 Community License Agreement, refer to the THIRD_PARTY_LICENSES file.

External Service Dependencies

This project integrates with external services that users install and manage separately:

• LM Studio: Users install LM Studio independently. Users are responsible for compliance with LM Studio's terms of service and the licenses of any models they download through LM Studio.
• OpenAI API: Users provide their own API keys and are responsible for compliance with OpenAI's terms of service.
• AWS Services: When deployed on AWS, users are responsible for compliance with AWS terms of service.

Attribution Notices

• Built with Meta Llama 3.3 (as required by Meta's license)
• Parakeet models by NVIDIA Corporation (as required by CC-BY-4.0)

Medical Disclaimer

[!IMPORTANT] The Licensed Work is provided as a support tool only and is not intended as a substitute for the guidance or care of a health professional.

[!CAUTION] The authors disclaim all warranties, expressed or implied. In particular, but without limitation, the Licensed Work is provided WITHOUT WARRANTY OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE, EITHER EXPRESSED OR IMPLIED. The user assumes all responsibility for losses, costs, claims, damages or liability of any kind whatsoever which may arise from use of the Licensed Work.