A self-hosted ML coding practice platform. 68 problems from ReLU to flow matching — attention, training, RLHF, diffusion, and more. Instant feedback in the browser.
Python
1,299
60 commits
updated May 12, 2026
Implement the internals of modern AI systems from scratch — Transformers, vLLM, TRL, and beyond.
Read the paper, then write the code. No GPU required.
76 problems. You write the implementation, a local grading service runs the tests, you see what broke. That's it.
The problems cover what's actually inside Transformers, vLLM, TRL, diffusion models, and GNNs — attention variants, training tricks, inference kernels, alignment algorithms, graph neural networks. No GPU needed.
nn.MultiheadAttention, now write it yourself.env or per-user in the UI)| Layer | Technology |
|---|---|
| Frontend | Next.js + Monaco Editor + Tailwind CSS |
| Backend | FastAPI grading service |
| Judge Engine | torch_judge — executes and validates submissions |
| Storage | SQLite (progress tracking) |
print() statements now show in test results.Option A — one-liner (recommended)
macOS / Linux:
git clone https://github.com/whwangovo/pyre-code.git
cd pyre-code
./setup.sh
npm run dev
Windows (PowerShell):
git clone https://github.com/whwangovo/pyre-code.git
cd pyre-code
.\setup.ps1
npm run dev
Windows (CMD):
git clone https://github.com/whwangovo/pyre-code.git
cd pyre-code
setup.bat
npm run dev
The setup script automatically creates a .venv Python environment (prefers uv, falls back to python -m venv), installs all dependencies, then prints the start command.
When .venv exists, npm run dev prefers that project-local Python automatically. If .venv is missing, it falls back to the current shell's python.
Option B — conda
git clone https://github.com/whwangovo/pyre-code.git
cd pyre-code
conda create -n pyre python=3.11 -y && conda activate pyre
pip install -e ".[dev]"
npm install
npm run dev # run with conda env activated
Option C — manual (venv)
git clone https://github.com/whwangovo/pyre-code.git
cd pyre-code
# create a Python env — pick one:
uv venv --python 3.11 .venv && source .venv/bin/activate && uv pip install -e ".[dev]"
# or: python3 -m venv .venv && source .venv/bin/activate && pip install -e ".[dev]"
# Windows: python -m venv .venv && .venv\Scripts\activate && pip install -e ".[dev]"
npm install
npm run dev
Either way, once running:
http://localhost:8000http://localhost:3000Option D — Docker
git clone https://github.com/whwangovo/pyre-code.git
cd pyre-code
docker compose up --build
Open http://localhost:3000. Progress is persisted in a Docker volume. Run docker compose down -v to reset.
To enable server-side AI hints, copy web/.env.example to web/.env and fill in:
AI_HELP_BASE_URL=https://api.openai.com/v1
AI_HELP_API_KEY=sk-...
AI_HELP_MODEL=gpt-4o-mini
Any OpenAI-compatible endpoint works (OpenAI, Anthropic via proxy, Ollama, etc.). Users can also configure their own API key in the UI if no server-side config is set.
76 problems organized by category:
| Category | Problems |
|---|---|
| Fundamentals | ReLU, Softmax, GELU, SwiGLU, Dropout, Embedding, Linear, Kaiming Init, Linear Regression |
| Normalization | LayerNorm, BatchNorm, RMSNorm |
| Attention | Scaled Dot-Product, Multi-Head, Causal, Cross, GQA, Sliding Window, Linear, Flash, Differential, MLA |
| Position Encoding | Sinusoidal PE, RoPE, ALiBi, NTK-aware RoPE |
| Architecture | SwiGLU MLP, GPT-2 Block, ViT Patch, ViT Block, Conv2D, Max Pool, Depthwise Conv, MoE, MoE Load Balance |
| Training | Adam, Cosine LR, Gradient Clipping, Gradient Accumulation, Mixed Precision, Activation Checkpointing |
| Distributed | Tensor Parallel, FSDP, Ring Attention |
| Inference | KV Cache, Top-k Sampling, Beam Search, Speculative Decoding, BPE, INT8 Quantization, Paged Attention |
| Loss & Alignment | Cross Entropy, Label Smoothing, Focal Loss, Contrastive Loss, DPO, GRPO, PPO, Reward Model |
| Diffusion & DiT | Noise Schedule, DDIM Step, Flow Matching, adaLN-Zero |
| Adaptation | LoRA, QLoRA |
| Reasoning | MCTS, Multi-Token Prediction |
| SSM | Mamba SSM |
| Graph Neural Networks | GCN, Graph Readout, GAT, GIN, MPNN, GraphSAGE, Link Prediction, Graph Autoencoder |
Pick one based on what you're working toward:
| Path | Problems | Description |
|---|---|---|
| Transformer Internals | 12 | Activations → Normalization → Attention → GPT-2 Block |
| Attention & Position Encoding | 13 | Every attention variant + RoPE, ALiBi, NTK-RoPE |
| Train a GPT from Scratch | 15 | Embeddings → architecture → loss → optimizer → training tricks |
| Inference & Distributed Training | 9 | KV cache, quantization, sampling, tensor parallel, FSDP |
| Alignment & Agent Reasoning | 6 | Reward model → DPO → GRPO → PPO → MCTS |
| Vision Transformer Pipeline | 7 | Conv → patch embedding → ViT block |
| Diffusion Models & DiT | 5 | Noise schedule → DDIM → flow matching → adaLN-Zero |
| LLM Frontier Architectures | 7 | GQA, Differential Attention, MLA, MoE, Multi-Token Prediction |
| Graph Neural Networks | 8 | GCN → GAT → GIN → MPNN → GraphSAGE → Link Prediction → GAE |
Not sure where to start?
Fundamentals ──→ Transformer Internals ──→ Train a GPT from Scratch
│ │
▼ ▼
Attention & Position Inference & Distributed
│ │
▼ ▼
LLM Frontier Archs Alignment & Reasoning
│
┌───────┼───────┐
▼ ▼ ▼
Vision Trans. Diffusion Graph Neural Networks
| Variable | Default | Description |
|---|---|---|
GRADING_SERVICE_URL | http://localhost:8000 | Grading service URL |
DB_PATH | ./data/pyre.db | SQLite database for progress tracking |
Set in web/.env.local to override.
pyre/
├── web/ # Next.js frontend
│ ├── src/app/ # Pages and API routes
│ ├── src/components/ # UI components
│ └── src/lib/ # Utilities, problem data
├── grading_service/ # FastAPI backend
├── torch_judge/ # Judge engine (problem definitions + test runner)
└── package.json # Dev scripts (runs frontend + backend concurrently)
Contributions are welcome! Here are some ways you can help:
torch_judge/Please open an issue first for larger changes so we can discuss the approach.
Problem set and judge engine based on TorchCode by duoan, licensed under MIT.
Distributed under the MIT License. See LICENSE for more information.
158 followers · starred May 2026
463 followers · starred Apr 2026
658 followers · starred Apr 2026
253 followers · starred May 2026
A self-hosted ML coding practice platform. 68 problems from ReLU to flow matching — attention, training, RLHF, diffusion, and more. Instant feedback in the browser.
Python
1,299
60 commits
updated May 12, 2026
Implement the internals of modern AI systems from scratch — Transformers, vLLM, TRL, and beyond.
Read the paper, then write the code. No GPU required.
76 problems. You write the implementation, a local grading service runs the tests, you see what broke. That's it.
The problems cover what's actually inside Transformers, vLLM, TRL, diffusion models, and GNNs — attention variants, training tricks, inference kernels, alignment algorithms, graph neural networks. No GPU needed.
nn.MultiheadAttention, now write it yourself.env or per-user in the UI)| Layer | Technology |
|---|---|
| Frontend | Next.js + Monaco Editor + Tailwind CSS |
| Backend | FastAPI grading service |
| Judge Engine | torch_judge — executes and validates submissions |
| Storage | SQLite (progress tracking) |
print() statements now show in test results.Option A — one-liner (recommended)
macOS / Linux:
git clone https://github.com/whwangovo/pyre-code.git
cd pyre-code
./setup.sh
npm run dev
Windows (PowerShell):
git clone https://github.com/whwangovo/pyre-code.git
cd pyre-code
.\setup.ps1
npm run dev
Windows (CMD):
git clone https://github.com/whwangovo/pyre-code.git
cd pyre-code
setup.bat
npm run dev
The setup script automatically creates a .venv Python environment (prefers uv, falls back to python -m venv), installs all dependencies, then prints the start command.
When .venv exists, npm run dev prefers that project-local Python automatically. If .venv is missing, it falls back to the current shell's python.
Option B — conda
git clone https://github.com/whwangovo/pyre-code.git
cd pyre-code
conda create -n pyre python=3.11 -y && conda activate pyre
pip install -e ".[dev]"
npm install
npm run dev # run with conda env activated
Option C — manual (venv)
git clone https://github.com/whwangovo/pyre-code.git
cd pyre-code
# create a Python env — pick one:
uv venv --python 3.11 .venv && source .venv/bin/activate && uv pip install -e ".[dev]"
# or: python3 -m venv .venv && source .venv/bin/activate && pip install -e ".[dev]"
# Windows: python -m venv .venv && .venv\Scripts\activate && pip install -e ".[dev]"
npm install
npm run dev
Either way, once running:
http://localhost:8000http://localhost:3000Option D — Docker
git clone https://github.com/whwangovo/pyre-code.git
cd pyre-code
docker compose up --build
Open http://localhost:3000. Progress is persisted in a Docker volume. Run docker compose down -v to reset.
To enable server-side AI hints, copy web/.env.example to web/.env and fill in:
AI_HELP_BASE_URL=https://api.openai.com/v1
AI_HELP_API_KEY=sk-...
AI_HELP_MODEL=gpt-4o-mini
Any OpenAI-compatible endpoint works (OpenAI, Anthropic via proxy, Ollama, etc.). Users can also configure their own API key in the UI if no server-side config is set.
76 problems organized by category:
| Category | Problems |
|---|---|
| Fundamentals | ReLU, Softmax, GELU, SwiGLU, Dropout, Embedding, Linear, Kaiming Init, Linear Regression |
| Normalization | LayerNorm, BatchNorm, RMSNorm |
| Attention | Scaled Dot-Product, Multi-Head, Causal, Cross, GQA, Sliding Window, Linear, Flash, Differential, MLA |
| Position Encoding | Sinusoidal PE, RoPE, ALiBi, NTK-aware RoPE |
| Architecture | SwiGLU MLP, GPT-2 Block, ViT Patch, ViT Block, Conv2D, Max Pool, Depthwise Conv, MoE, MoE Load Balance |
| Training | Adam, Cosine LR, Gradient Clipping, Gradient Accumulation, Mixed Precision, Activation Checkpointing |
| Distributed | Tensor Parallel, FSDP, Ring Attention |
| Inference | KV Cache, Top-k Sampling, Beam Search, Speculative Decoding, BPE, INT8 Quantization, Paged Attention |
| Loss & Alignment | Cross Entropy, Label Smoothing, Focal Loss, Contrastive Loss, DPO, GRPO, PPO, Reward Model |
| Diffusion & DiT | Noise Schedule, DDIM Step, Flow Matching, adaLN-Zero |
| Adaptation | LoRA, QLoRA |
| Reasoning | MCTS, Multi-Token Prediction |
| SSM | Mamba SSM |
| Graph Neural Networks | GCN, Graph Readout, GAT, GIN, MPNN, GraphSAGE, Link Prediction, Graph Autoencoder |
Pick one based on what you're working toward:
| Path | Problems | Description |
|---|---|---|
| Transformer Internals | 12 | Activations → Normalization → Attention → GPT-2 Block |
| Attention & Position Encoding | 13 | Every attention variant + RoPE, ALiBi, NTK-RoPE |
| Train a GPT from Scratch | 15 | Embeddings → architecture → loss → optimizer → training tricks |
| Inference & Distributed Training | 9 | KV cache, quantization, sampling, tensor parallel, FSDP |
| Alignment & Agent Reasoning | 6 | Reward model → DPO → GRPO → PPO → MCTS |
| Vision Transformer Pipeline | 7 | Conv → patch embedding → ViT block |
| Diffusion Models & DiT | 5 | Noise schedule → DDIM → flow matching → adaLN-Zero |
| LLM Frontier Architectures | 7 | GQA, Differential Attention, MLA, MoE, Multi-Token Prediction |
| Graph Neural Networks | 8 | GCN → GAT → GIN → MPNN → GraphSAGE → Link Prediction → GAE |
Not sure where to start?
Fundamentals ──→ Transformer Internals ──→ Train a GPT from Scratch
│ │
▼ ▼
Attention & Position Inference & Distributed
│ │
▼ ▼
LLM Frontier Archs Alignment & Reasoning
│
┌───────┼───────┐
▼ ▼ ▼
Vision Trans. Diffusion Graph Neural Networks
| Variable | Default | Description |
|---|---|---|
GRADING_SERVICE_URL | http://localhost:8000 | Grading service URL |
DB_PATH | ./data/pyre.db | SQLite database for progress tracking |
Set in web/.env.local to override.
pyre/
├── web/ # Next.js frontend
│ ├── src/app/ # Pages and API routes
│ ├── src/components/ # UI components
│ └── src/lib/ # Utilities, problem data
├── grading_service/ # FastAPI backend
├── torch_judge/ # Judge engine (problem definitions + test runner)
└── package.json # Dev scripts (runs frontend + backend concurrently)
Contributions are welcome! Here are some ways you can help:
torch_judge/Please open an issue first for larger changes so we can discuss the approach.
Problem set and judge engine based on TorchCode by duoan, licensed under MIT.
Distributed under the MIT License. See LICENSE for more information.
158 followers · starred May 2026
463 followers · starred Apr 2026
658 followers · starred Apr 2026
253 followers · starred May 2026