LAtelier

A multi-page local AI workbench. Each feature is a self-contained page that plugs into a shared framework — sidebar, global task queue, status bar, GPU lifecycle, styling — so adding a new tool costs almost no extra code.

Currently ships with:

• ⚔ LLM Duel — side-by-side comparison of two local Gemma models with configurable quantization.
• 📄 PDF Parser — PDF → Markdown via Marker.
• ⚙ Settings — system prompt + sampling sliders.
• 📊 Performance — live GPU usage, task queue with cancel buttons, history, per-run log.
• ℹ About — project info + model/quant compatibility matrix.

Built on Google Mesop.

---

Contents

1. Installation
2. Running
3. Repository layout
4. Architecture overview
5. Adding a new page in 4 steps
6. Framework conventions (things every page must follow)

---

Installation

Hardware

Tested on 2× NVIDIA RTX 3090 (24 GB each), CUDA 13, Ubuntu 24. A single 24 GB GPU can run NF4 12B but not BF16 12B. The 25.2B MoE requires both GPUs + CPU offload.

Main app (venv/)

git clone https://github.com/fliaght/latelier.git
cd latelier

python3 -m venv venv
source venv/bin/activate
pip install --upgrade pip
pip install -r requirements.txt

PDF Parser backend (venv_marker/) — optional

Marker has its own PyTorch + Surya models and would otherwise downgrade the main venv's transformers. Keep it separate:

python3 -m venv venv_marker
source venv_marker/bin/activate
pip install --upgrade pip
pip install -r requirements-marker.txt

Marker models auto-download into ~/.cache/ on first parse (several hundred MB).

---

Running

./restart_mesop.sh

The script:

1. Kills any previous mesop app_mesop.py process (SIGTERM, then SIGKILL after 2 s).
2. Polls nvidia-smi until VRAM is below 1 GB.
3. Starts Mesop bound to 0.0.0.0:7861 with unbuffered stdout.

Open http://localhost:7861 (SSH-forward the port if remote).

Stop cleanly with pkill -TERM -f "mesop app_mesop.py". SIGTERM, SIGINT, and atexit handlers in app_mesop.py all call engine.clear_gpu() before the process dies.

---

Repository layout

latelier/
├── app_mesop.py              # Entry: monkey-patches, @me.page, exit handlers
│
├── state.py                  # @me.stateclass State + SESSION_CACHE helpers
├── styles.py                 # Color tokens + style factories
├── tasks.py                  # TaskManager, @task, EarlyExit
├── shell.py                  # Page dataclass, sidebar, status, render_shell
├── engine.py                 # HuggingFace lifecycle (load/clear, tokenizers)
│
├── pages/
│   ├── __init__.py           # PAGES tuple (registry)
│   ├── duel.py               # ⚔ LLM Duel
│   ├── pdf_parse.py          # 📄 PDF Parser (Marker subprocess)
│   ├── settings.py           # ⚙ Settings
│   ├── performance.py        # 📊 Performance (queue + log + resources)
│   └── about.py              # ℹ About
│
├── requirements.txt          # Main app deps
├── requirements-marker.txt   # PDF Parser backend deps (separate venv)
├── restart_mesop.sh          # Safe restart (kill + VRAM poll + start)
│
└── README.md                 # This file

archive/, reports/, CLAUDE.md, venv/, venv_marker/ are git-ignored.

---

Architecture overview

Three concerns are strictly separated and imports flow in one direction only.

┌──────────────────────────────────────────────────────────────┐
│  app_mesop.py          Entry: @me.page + signal handlers    │
└──────────────────────────────────────────────────────────────┘
                              │
                              ▼
┌───────────── STYLES ─────────────┐   ┌───────────── FRAMEWORK ──────────────┐
│ styles.py                        │   │ state.py    — @me.stateclass State   │
│   C (color tokens)               │   │              + SESSION_CACHE helpers │
│   card_style / page_container /  │   │                                      │
│   page_header / section_header   │   │ tasks.py    — TaskManager, @task,    │
│   edge_button_style              │   │              EarlyExit, FIFO queue   │
└──────────────────────────────────┘   │                                      │
           ▲                           │ shell.py    — Page type, sidebar,    │
           │                           │              floating status,        │
           │                           │              render_shell(pages)     │
           │                           │                                      │
           │                           │ engine.py   — HuggingFace models:    │
           │                           │              MODELS, tokenizers,     │
           │                           │              load_model, clear_gpu  │
           │                           └──────────────────────────────────────┘
           │                                        ▲
           └────────────────────────────────────────┘
                              │
                              ▼
┌───────────── PAGES (one module each) ────────────────────┐
│  pages/duel.py, pdf_parse.py, settings.py,               │
│  performance.py, about.py                                │
│  pages/__init__.py  — PAGES = (...,)                     │
└──────────────────────────────────────────────────────────┘

• Pages may import from shell, state, styles, tasks, engine.
• Shell may import from state, styles, tasks.
• Tasks may import from state.
• Styles / state are leaves.
• The framework never imports from pages/ (that would cycle). shell.render_shell(pages) takes the registered tuple as an argument.

Request flow (multi-user)

click "Send"  → pages/duel.py::_on_send
              → yield from _run_duel()
              → @task wrapper submits QueueEntry, sets is_busy=True
              → polls TASK_MANAGER.try_start() every 0.3 s (yields)
              → when next: load LEFT, stream, unload; load RIGHT, stream, unload
              → TASK_MANAGER.finish()
              → is_busy=False, final yield → sidebar re-enables

At most one task runs across the whole process. Other sessions submitting tasks see "Queued (position N)..." until their turn.

---

Adding a new page in 4 steps

1. Create pages/<name>.py

import mesop as me

from shell import Page
from state import State
from styles import card_style, page_container_style, page_header, page_inner_style

def _render():
    state = me.state(State)
    with me.box(style=page_container_style()):
        with me.box(style=page_inner_style()):
            page_header("My Page", "Short description.")
            with me.box(style=card_style()):
                me.text("Content here.")

PAGE = Page(key="my", title="My Page", icon="✨", render=_render)

2. For any long-running task, wrap it with @task

from tasks import EarlyExit, task

@task("My Work")
def _run_work(cancel_event):
    state = me.state(State)
    state.status = "Step 1..."
    yield
    if cancel_event.is_set():
        raise EarlyExit()
    # ... do work, yield periodically, check cancel_event ...

Event handlers invoke the worker via yield from:

def _on_start(e: me.ClickEvent):
    yield from _run_work()

3. If the task produces a large result, cache it (don't put it on State)

from state import session_get, session_set

KEY_MY_OUTPUT = "my.output"

def _render():
    state = me.state(State)
    output = session_get(state, KEY_MY_OUTPUT, "") or ""
    me.markdown(output)

4. Register in pages/__init__.py

from . import duel, pdf_parse, my_page, settings, performance, about

PAGES = (
    duel.PAGE,
    pdf_parse.PAGE,
    my_page.PAGE,
    settings.PAGE,
    performance.PAGE,
    about.PAGE,
)

Done. The sidebar picks up the new nav item automatically, the floating status pill shows state.status, the Performance page shows your task entries with working Cancel buttons, and all Mesop rendering gotchas are handled by the shared helpers.

---

Framework conventions (things every page must follow)

What you never reimplement:

• Sidebar navigation, nav styling, busy-state disabling
• Floating status pill
• Global queueing, per-session mutex, cancellation plumbing
• GPU cleanup (engine.clear_gpu handles accelerate's quirks)
• Mesop DOM-reuse workarounds (shell helpers already use state-encoded keys)

Hard rules to keep the framework sustainable

1. Never capture state in a module-level closure — get it inside each event handler with me.state(State).
2. Event handlers are plain named module-level functions — no lambda, no functools.partial, no closure factories. Mesop identifies handlers by __qualname__ and collapses same-named ones.
3. Keep State tiny — only primitive fields. Anything ≥ 1 KB goes into SESSION_CACHE via session_get / session_set.
4. Interactive elements whose behavior depends on state need a state-encoded key — e.g., key=f"send-btn-{'busy' if state.is_busy else 'ready'}".
5. In a @task body, use EarlyExit — never return — return skips the decorator's post-finally re-render, leaving the sidebar stuck disabled.
6. Dependency direction is one-way: pages → (shell/state/styles/tasks/engine). Nothing in the framework imports from pages/.
7. After any code change, restart via ./restart_mesop.sh — Mesop's hot reload is not reliable.

---

License

GPL-3.0-or-later. See LICENSE for the full text.

Use, modify, and redistribute freely; if you distribute a modified version (including over a network), you must release your modifications under the same license.