Laya Plays Magics

An experimental Magic: The Gathering Arena draft companion for The Hobbit (HOB), Premier Draft. It uses two fine-tuned Laya models to suggest picks and build decks, plus LightGBM models to estimate deck win probability.

How it works

The desktop app reads cards manually or through Arena screenshots with PaddleOCR. Laya selects a card from the current pack using the drafted pool, then proposes maindeck copy counts after the draft. You review the result, adjust it to 40 cards, evaluate it, and save the run in SQLite.

Inference runs locally in Python sidecars managed by the app. Laya returns typed decisions and probabilities; card facts come from MTGJSON. The models imitate human decisions and do not guarantee better picks, legal 40-card decks, or wins.

Stack: Wails v2 / Go, React 19 / TypeScript / Vite, Tailwind CSS 4, TanStack Store, i18next, SQLite / GORM, Laya / PyTorch, FastAPI, PaddleOCR, and LightGBM. The UI supports English and Brazilian Portuguese.

Models

Download the repositories into those directories using an authorized Hugging Face account if required; unauthenticated checks returned HTTP 401. Each Laya checkpoint needs model.safetensors, rl_agent_config.json, encoder/, and tokenizer/.

The app uses models/artifacts/hob_deck_evaluator_lightgbm_v2/, containing model.txt, feature_names.json, calibration.json, and metrics.json. The evaluator repository also describes v4; downloading it does not switch the app from v2.

Development

Run commands from the repository root. The main development target is macOS Apple Silicon. Install Go 1.25+, Node.js compatible with Vite 7, Python 3.12, and the Wails platform prerequisites.

go install github.com/wailsapp/wails/v2/cmd/wails@v2.16.0
npm run install:app
npm run doctor

mkdir -p "$HOME/.cache/laya-plays-magics"
python3.12 -m venv "$HOME/.cache/laya-plays-magics/sidecar-venv"
"$HOME/.cache/laya-plays-magics/sidecar-venv/bin/pip" install -r requirements/sidecar-laya.txt

python3.12 -m venv "$HOME/.cache/laya-plays-magics/ocr-venv"
"$HOME/.cache/laya-plays-magics/ocr-venv/bin/pip" install -r requirements/sidecar-ocr.txt
export PADDLEOCR_PYTHON="$HOME/.cache/laya-plays-magics/ocr-venv/bin/python"

npm run dev

Download the models first and ensure assets/mtgjson/HOB.json exists. Set PADDLEOCR_PYTHON again in new shells. The app starts inference automatically; no separate server is needed. OCR downloads its models on first use and needs macOS Screen Recording permission with Arena visible.

Keep the root scripts' build/bin symlink: it places macOS output on APFS because exFAT AppleDouble files break code signing. See application notes for platform details and environment overrides.

Training data

Download and decompress the matching HOB / PremierDraft draft and game CSVs from 17Lands public datasets. Obtain the set JSON from MTGJSON:

assets/17lands/draft_data_public.HOB.PremierDraft.csv
assets/17lands/game_data_public.HOB.PremierDraft.csv
assets/mtgjson/HOB.json

Draft rows supply packs, pools, picks, and historical player buckets. Game rows supply registered decks, sideboards, and outcomes. Preserve download dates, hashes, and MTGJSON export metadata; changing snapshots can change results. Do not mix expansions or event types.

Generate the two Laya JSONL datasets

Install tooling and select 3,000 complete drafts from players with at least 100 historical games and a win-rate bucket of at least 58%:

npm run setup:py
.venv/bin/pip install tokenizers

.venv/bin/python tools/dataset/select_draft_dataset.py \
  --n-drafts 3000 --min-games-bucket 100 --min-winrate 0.58 --seed 42

Generate pick decisions and deckbuilding decisions using the default source paths above:

.venv/bin/python tools/dataset/build_pick_jsonl.py \
  --val-fraction 0.1 --test-fraction 0.1 --head-max-len 768

.venv/bin/python tools/dataset/build_deck_jsonl.py \
  --val-fraction 0.1 --test-fraction 0.1 \
  --deck-size 40 --basic-lands questions --max-basic-copies 17 \
  --head-max-len 768 --max-len 1280

Outputs in assets/final/:

• draft_data_HOB_PremierDraft_filtered_3000.jsonl: one choice question per pick, labeled with the human selection.
• draft_data_HOB_PremierDraft_filtered_3000_deckbuild.jsonl: one record per registered build, with copy-count questions for pool cards and five basic lands.

The builders join records by draft ID, validate card/build consistency, and preserve provenance. Splits keep each draft together. In each JSONL row, state, questions, and gold are JSON-encoded strings; decode them with json.loads. Inference accepts decoded state and questions only.

Fine-tune Laya

Use the official fine-tuning notebook, adapting its dataset loader for each JSONL in separate jobs. This repository does not include the complete original MTG training notebook.

Start from convaiinnovations/laya, use max_len=1280 and head_max_len=768 consistently in preprocessing, training, and exported configuration. Reserve separate training, validation, calibration, and test drafts before expanding builds into questions; replace the notebook's item-level calibration sampling accordingly. Select checkpoints on validation and calibrate only on the calibration partition.

The recorded runs used four epochs: pick epoch 4 reached 69.1% validation agreement; deckbuild epoch 3 reached 79.56%. These measure imitation, not gameplay improvement. Deckbuild probabilities remain uncalibrated and its full test benchmark was incomplete. See the pick and deckbuild cards for details.

Deck evaluators

These are separate LightGBM models over maindeck card-copy counts, not Laya models. v1 uses the deckbuild JSONL joined to games; v2 uses raw CSVs filtered by player history; v3 selects the best half by historical buckets; v4 uses all compatible HOB drafts.

For v2–v4, wins/losses are aggregated per 40-card build and used as weighted binary targets. Draft-level splits prevent overlap; validation is divided between early stopping and isotonic calibration. Game-time information and player skill are not features.

.venv/bin/pip install -r requirements/evaluator.txt

# App's evaluator; replace v2 with v3 or v4 for other experiments.
.venv/bin/python tools/deck_evaluator/build_hob_deck_evaluator_v2_jsonl.py
.venv/bin/python tools/deck_evaluator/train_hob_deck_evaluator_v2.py

# my_deck.json: exact card names mapped to counts, totaling 40.
.venv/bin/python tools/deck_evaluator/evaluate_hob_deck_v2.py \
  --deck-json my_deck.json

Artifacts go to models/artifacts/hob_deck_evaluator_lightgbm_v2/. On macOS, install libomp if LightGBM requires it. Keep this environment separate from the Laya sidecar to avoid OpenMP conflicts; the app uses pure-Python inference for its saved v2 trees.

Predictive signal is weak: roughly 0.53 test AUC for v2 and 0.57 for v4. Scores are estimates, not promised win rates. See the evaluator card and experiment notes.

Standalone servers

For experiments and benchmarks, start both Laya servers separately from the app:

LAYA_PYTHON="$HOME/.cache/laya-plays-magics/sidecar-venv/bin/python" npm run serve:hob

Picks use 127.0.0.1:8422; deckbuilding uses 127.0.0.1:8423. Both expose /health, /decide, and /docs. Ctrl+C stops them. Payload details: pick API, deckbuild API. Historical LAN addresses in those notes are not hosted services.

Production builds

npm run build          # Production desktop app only
npm run package:mac    # macOS arm64 DMG with Python sidecars and models

For packaging, download both Laya checkpoints and evaluator v2, load both checkpoints once in development, and run OCR once to cache its models. The installer script freezes sidecars, stages models with a SHA-256 manifest, builds the app, and applies ad-hoc signing. It does not perform Developer ID signing/notarization.

The DMG and checksum go to ~/.cache/laya-plays-magics/release-mac/. Override MODELS_SRC or PADDLEOCR_SRC for other model locations. The version comes from apps/MTGClientApp/wails.json.

Windows packaging must run on Windows with Python 3.12, Go, Node, Wails prerequisites, and Inno Setup 6:

powershell -ExecutionPolicy Bypass -File packaging\windows\build_windows.ps1 -ModelsSrc D:\models -PaddleOcrSrc D:\official_models

Windows packaging and real Arena capture still require target-machine validation. See packaging instructions. Installed users do not need Python, Node, or Go; data is stored in ~/Library/Application Support/MTGClientApp on macOS or %APPDATA%\MTGClientApp on Windows.

Tests and project layout

npm test             # Python
npm run test:web     # Frontend
npm run test:go      # Go backend
npm run test:ci      # All suites with coverage gates

Application code lives in apps/MTGClientApp/; dataset builders in tools/dataset/; evaluator code in tools/deck_evaluator/; serving and benchmarks in their corresponding tools/ directories. Detailed guides are in docs/.

For troubleshooting, check apps/MTGClientApp/assets/logs/laya-sidecar.log. Override model paths with MTGCLIENT_MODELS_DIR or the device with LAYA_DEVICE. After changing deckbuilding weights, invalidate the DeckDecision cache: its key does not include the model version. Back up the database first.

Credits

• 17Lands and its contributing players: source of draft choices, registered decks, and game outcomes. Model cards identify the data as CC BY 4.0; retain attribution and check current source terms. This project filters, joins, and reformats those records.
• Laya / Convai Innovations and maintainers: upstream architecture, base checkpoint, SDK, and training implementation, published under Apache-2.0.
• MTGJSON supplies card facts; Scryfall supplies card imagery. Magic: The Gathering assets and marks belong to their respective rights holders, including Wizards of the Coast.

This is an independent experimental project. Third-party data, models, artwork, and dependencies retain their own licenses; their inclusion does not imply endorsement.