durable-media-pipeline

A multi-stage media generation pipeline built to survive interruption. The whole run is a state machine persisted to SQLite, so a crash, a reboot or a killed GPU job resumes from the last completed stage instead of starting over. A stage that raises records its traceback against the job without changing the job's state, so the next scheduled run picks it up again rather than skipping it.

The pipeline itself goes seed title -> LLM-written script -> TTS narration -> word-level timestamps -> AI panel images -> FFmpeg assembly -> operator review over Telegram -> scheduled upload. Nothing publishes without a human approving it.

The interesting parts of this repo are the crash-safe state machine, the operator-in-the-loop review gate, the two-stage long-form-to-shorts extraction, and the fact that it runs GPU-bound work on consumer hardware unattended. It is not a research project - it has shipped real videos on a schedule.

Originally built for @Vision-m1i, a faceless shorts channel that later pivoted to historical biographies. The pipeline is niche-agnostic: swap the seed bank, style suffix, voice preset and prompt templates and it produces a different channel.

---

Pipeline

Seed bank title
  -> Anthropic Claude script (long-form + short-clip extraction)
  -> Kokoro TTS narration
  -> whisper-timestamped word-level alignment
  -> Stable Diffusion XL panel generation (with optional style LoRA)
  -> FFmpeg assembly (panels + narration + ambient bed + subtitles)
  -> Telegram review (Approve / Edit Title / Reject)
  -> YouTube scheduled upload

State machine, persisted in data/jobs.sqlite:

scraped -> scripted -> narrated -> paneled -> assembled
        -> awaiting_review -> approved | rejected -> posted

A crashed run resumes at the stage that failed, so completed stages are not re-done. Within a stage the work is repeated rather than skipped, with one exception: extract_short is idempotent, and re-running it on a parent that already has children is a no-op. The DB is also the single source of truth for scheduling (scheduled_for), retries, and posted-video stats.

---

Why publish it

A complete content-generation pipeline, end-to-end, that runs unattended on a single Windows box. Useful as reference for anyone building:

• A long-form video generation pipeline on consumer GPU hardware
• A state-machine driven media pipeline with crash-safe resume
• An operator-in-the-loop review system over Telegram bot inline buttons
• A YouTube Data API uploader with scheduled-publish + thumbnail + end-screen
• A long-form-to-shorts extractor that finds the best 60-second clips inside a 10-minute video

This was production code for a real channel. The repo includes the full test suite - 123 test functions across 22 files, counted with grep -rE '^\s*(async )?def test_' tests/ - covering the assemble stage, telegram bot, seed bank dedup, prompts, scheduled-slot picker, end-screen render, thumbnail render and YouTube upload, so you can see how every stage is exercised.

---

Module map

---

Key technical decisions worth a read

1. State machine in SQLite, resumable stages

Every stage reads its input from the DB and writes its output back, and the job's state only advances when a stage completes. A stage that raises records its traceback against the job and leaves the state untouched (db.transition(..., state=None, error=...)), so the next scheduled run retries that stage instead of the pipeline stalling or silently skipping ahead.

Resume is at stage granularity, not within a stage. If panel-gen dies at panel 18 of 20, the re-run regenerates all 20. What makes that survivable rather than merely slow is that the seed is deterministic: _seed_for(job_id, panel_index) is a sha256 of the job id and the panel index, not Python's per-interpreter salted hash(), so a re-run reproduces the same images rather than a different set. Panel-gen on a 10-minute long-form can take 25+ minutes on one GPU, so this is the stage where that distinction costs the most.

2. Long-form-to-shorts extraction

The pipeline produces 2 long-form videos a week and 4 shorts a week. The shorts are not generated independently - they're extracted from the long-form videos by extract_short.py, which uses two complementary heuristics:

• Cold-open clip: the first ~60 seconds of the long-form, lightly edited to stand alone
• Peak-moment clip: a 60-second window centered on the highest-energy moment of the script (detected by sentiment-spike heuristic on the script tokens)

This means each long-form generates ~2 shorts of compatible content, all sharing thumbnail style and brand. The shorts act as a discovery funnel; the long-form is where retention + watch-hours accumulate (YPP requirement). One pipeline run produces both.

3. Operator-in-the-loop, not fully automated

Each generated video pings a Telegram bot with Approve / Edit Title / Reject buttons. The operator approves in under 30 seconds per video, 5 videos a day = under 10 minutes of human review. This is the right amount of automation: full hands-off generation is too risky (one bad title = one strike), and full manual generation is too slow.

The auto_approve config flag also exists for fully unattended mode - the bot still pings, but the slot is filled and the video uploaded automatically. Used once trust is established.

4. GPU pipeline on a single consumer card

The whole pipeline (Kokoro TTS, whisper-timestamped, SDXL panel-gen) runs sequentially on a single 12 GB+ GPU. Panel-gen at 28 steps + SDXL base + LoRA fits in 11 GB, leaving headroom for Whisper. The Blackwell card (RTX 5070, sm_120) needs CUDA 12.8+ - earlier wheels are pinned out via pyproject.toml's [tool.uv.sources].

5. uv + frozen lockfile across hosts

uv.lock is committed. Fresh clones run uv sync --frozen to install exactly what's in the lockfile. This is the only thing that makes the GPU dependency stack reproducible across hosts.

---

Setup

1. System prereqs

• Windows 11 with PowerShell as Admin
• NVIDIA RTX 5070 (or other 12 GB+ Blackwell/Ada GPU) with current drivers
• ffmpeg and ffprobe on PATH (winget install Gyan.FFmpeg works)
• Git, Python 3.11, uv installed

2. Clone + install

git clone https://github.com/visione4906/durable-media-pipeline.git
cd durable-media-pipeline
uv sync --frozen
uv run python -c "import torch; assert torch.cuda.is_available(); x=torch.randn(1024,1024,device='cuda'); print('kernel ok:', (x@x.T).shape)"

torch is pinned to the cu128 wheel index in pyproject.toml. The RTX 5070 (sm_120 Blackwell) needs CUDA 12.8+; earlier wheels error out at runtime.

3. Drop in operator-supplied assets

• data/style_lora.safetensors - optional style LoRA from CivitAI or similar (SDXL-compatible). The pipeline runs without this file; panels just come out in the base SDXL style (less consistent).
• data/ambient_bed.mp3 - royalty-free ambient track (loopable, ~2 min). Pixabay or YouTube Audio Library both have suitable options.

4. Configure secrets

Create secrets/ (gitignored) and populate:

• secrets/anthropic_key - Anthropic API key (one line, no JSON)
• secrets/telegram_bot_token - bot token from @BotFather (one line)
• secrets/telegram_chat.json - {"operator_chat_id": <your-numeric-chat-id>} (DM the bot once and read it from https://api.telegram.org/bot<TOKEN>/getUpdates)
• secrets/youtube_oauth.json - Google Cloud OAuth client secrets (Desktop type)
• Run uv run python scripts/bootstrap_oauth.py once to generate secrets/youtube_token.json

5. Initialise DB

uv run python -c "from horror_shorts.db import init_db, connect; from horror_shorts.config import DB_PATH, seed_default_config; init_db(DB_PATH); seed_default_config(connect(DB_PATH)); print('DB ready')"

6. Smoke tests

uv run pytest -v

Expected: every test passes, with the GPU-gated ones reported as skipped. The skipped set is the smoke tests for narrate, timestamps and panel-gen plus the assemble integration - they need a GPU, so they are opt-in. To run them too:

$env:RUN_GPU_TESTS=1
uv run pytest tests/test_narrate.py tests/test_timestamps.py tests/test_panel_gen.py -v

7. Install Windows scheduled tasks (PowerShell as Administrator)

.\scripts\install_task_scheduler.ps1

Six tasks are created:

• Vision-Generation - daily, runs the generation pipeline
• Vision-Uploader - every 15 min, uploads approved videos at their scheduled slots
• Vision-Housekeeping - daily, GCs old working dirs + alerts on stale jobs
• Vision-Poller - every 6 hours, polls view counts on posted videos
• Vision-ShortExtraction - Wednesdays and Fridays 02:00, extracts two vertical clips from the most recent posted long-form
• Vision-TelegramBot - runs continuously from logon (long-polling for review callbacks)

---

Configuration tuning

DB-table-tunable (no code change, no restart):

uv run python -c "from horror_shorts.db import connect; from horror_shorts.config import DB_PATH, set_config; set_config(connect(DB_PATH), 'posting_slots', '13:00,16:00,19:00,21:00,23:30')"

Available config keys: posting_slots, daily_target, youtube_handle, long_form_post_time_utc, long_form_post_days, short_clip_post_time_utc, short_clip_post_days, auto_approve.

---

Known caveats

Timezone - slot times are interpreted as UTC

scheduled_for values stored by the slot scheduler use the host's local naive datetime.now(), but the YouTube uploader treats those naive ISO strings as UTC when building the publishAt field. If you configure posting_slots = "14:00,17:00,...", videos publish at 14:00 / 17:00 UTC, not 14:00 / 17:00 local.

For a UK operator on BST (UTC+1) wanting 17:00 BST publish times, configure posting_slots = "16:00,..." (subtract 1h in summer, no offset in winter).

Voice (am_michael) is locked

This is the channel's voice identity. Don't change it without a deliberate brand decision - videos already published establishing the voice association will be wasted on a rebrand.

Stable Diffusion XL gating

Some SDXL forks are gated on Hugging Face - operator must accept the license terms once and then huggingface-cli login with a token that has read access. First panel-gen run will fail with a 403 until this is done.

---

What is NOT in this repo

Deliberately excluded:

• The actual secrets/ directory (operator-specific credentials)
• The style LoRA + ambient bed (licensing varies; bring your own)
• The Vision channel's specific brand assets beyond the logo/watermark/banner (those are kept as reference for the pipeline's outputs)
• The data/jobs.sqlite snapshot (this is runtime state, regenerated on first DB init)
• The OneDrive design SPEC (private)

---

License

MIT.

About

Built by Sal (Abdulsalam Oladapo). UK-based. Full profile at https://consentleads.uk/sal.