AI Story Maker

Generate AI video stories from text prompts. This tool takes your idea, expands it into a narrative, fetches beautiful images from Unsplash, converts the story to speech, and combines everything into a cinematic video with Ken Burns effects and subtitles.

Features

• Story Generation - Local LLM expands prompts into narratives with scene descriptions
• Unsplash Images - Fetches high-quality images matching your story scenes
• Ken Burns Effect - Adds cinematic pan/zoom animations to images
• Text-to-Speech - VibeVoice-Realtime-0.5B for high-quality voice synthesis
• Video Assembly - Combines images with audio, transitions, and subtitles
• Audio-Matched Duration - Each scene matches the voiceover length

Requirements

• Python 3.10+
• Unsplash API key (free at https://unsplash.com/developers)
• ~2GB disk space for TTS models

Installation

1. Clone/Navigate to the project

cd ai-story-maker

2. Create a virtual environment

python -m venv venv
source venv/bin/activate  # Linux/Mac

3. Install PyTorch

For Windows with AMD GPU (DirectML):

pip install torch torchvision torchaudio
pip install torch-directml

For Linux with AMD GPU (ROCm):

pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/rocm6.2

For NVIDIA GPU (CUDA):

pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu121

4. Install dependencies

pip install -r requirements.txt

5. Install ImageMagick (for subtitles)

sudo apt install imagemagick

Usage

Interactive mode

python main.py -i

Command line mode

python main.py "A brave knight discovers a hidden dragon in an enchanted forest"

Options

# Specify number of scenes (default: 3)
python main.py --scenes 5 "Space exploration adventure"

# Custom output path
python main.py -o my_video.mp4 "A peaceful morning in a Japanese garden"

Examples

# Fantasy story
python main.py "A wizard's apprentice accidentally opens a portal to another dimension"

# Nature documentary style
python main.py "The journey of a salmon swimming upstream"

# Sci-fi adventure
python main.py "First contact with an alien civilization on Mars"

Output

• Videos are saved to the output/ folder
• Default filename: story_YYYYMMDD_HHMMSS.mp4
• Format: MP4 (H.264 video, AAC audio)
• Resolution: 720x480

Configuration

Create a .env file with your settings:

# Required: Unsplash API key
UNSPLASH_ACCESS_KEY=your_unsplash_access_key_here

# Optional: Use a different LLM model for story generation
LLM_MODEL=google/flan-t5-large

First Run

The first run downloads AI models (~3GB total):

• VibeVoice-Realtime-0.5B (~2GB) - or use local vibe-voice/ folder
• LLM model (~500MB)

Models are cached in ~/.cache/huggingface/.

Pipeline Architecture

User Prompt
    ↓
┌─────────────────────────────────────┐
│  Step 1: Story Generation (LLM)    │
│  google/flan-t5-base               │
└─────────────────────────────────────┘
    ↓
┌─────────────────────────────────────┐
│  Step 2: Keyword Extraction        │
│  Extract scene keywords from story │
└─────────────────────────────────────┘
    ↓
┌─────────────────────────────────────┐
│  Step 3: Image Fetching            │
│  Unsplash API (high-quality imgs)  │
└─────────────────────────────────────┘
    ↓
┌─────────────────────────────────────┐
│  Step 4: Text-to-Speech            │
│  VibeVoice-Realtime-0.5B           │
└─────────────────────────────────────┘
    ↓
┌─────────────────────────────────────┐
│  Step 5: Video Assembly            │
│  Ken Burns effect + audio + subs   │
└─────────────────────────────────────┘
    ↓
Final Video (MP4)

Troubleshooting

Unsplash API errors

• Ensure UNSPLASH_ACCESS_KEY is set in .env
• Check API rate limits (50 requests/hour for free tier)
• Verify your API key at https://unsplash.com/oauth/applications

No images found

• Try more generic keywords
• Check your internet connection

Slow generation

• TTS is the main bottleneck (~10 seconds for most stories)
• First run downloads models (~2GB)

Project Structure

ai-story-maker/
├── main.py              # CLI entry point
├── config.py            # Configuration
├── story_generator.py   # LLM story + keyword extraction
├── image_fetcher.py     # Unsplash image fetching
├── video_creator.py     # Video assembly with Ken Burns
├── tts_engine.py        # Text-to-speech engine
├── vibevoice_tts.py     # VibeVoice model interface
├── vibe-voice/          # Local VibeVoice model (optional)
├── requirements.txt     # Dependencies
├── output/              # Generated videos
└── temp/                # Temporary files

Credits

• Unsplash - High-quality images
• VibeVoice - Text-to-speech by Microsoft
• moviepy - Video processing

License

MIT