Ropedia Academy — interactive bilingual learning app for egocentric vision, 3D reconstruction, human modeling, and world models.
3
stars
112
commits
Jupyter Notebook
primary language
Jul 16, 2026
updated
🌐 Live site · 4 tracks · 36 bilingual lessons · live 3D demos · spaced repetition
An interactive, bilingual (中文 / English) course on embodied & spatial AI — four connected tracks covering human modeling & motion, 3D/4D reconstruction & neural rendering, egocentric vision & interaction, and scene reconstruction & world models. Read lessons, play with live interactive demos (including real-time 3D), run the code in Colab, self-test, and review with spaced repetition. Runs entirely in the browser — no account required.
Beyond the per-lesson snippets, notebooks/training/ holds
twenty-three real, multi-cell Colab notebooks you can actually train — split into
clear blocks (data · model · train · compare) so you can step through and watch
each stage:
| Track | From scratch (PyTorch) | Foundation model |
|---|---|---|
| A · Human | SMPLify body fit · motion diffusion (DDPM) · 2D pose (heatmap) · 6D vs Euler rotation | — |
| B · 3D / rendering | NeRF (tiny_nerf) · neural SDF · 2D Gaussian Splatting · hash grid (Instant-NGP) · ICP registration · MAE pretraining | — |
| C · Egocentric | action anticipation (LSTM) · SimCLR self-supervised pretraining | CLIP probe · fine-tune VideoMAE · DINOv2 features |
| D · Scene / world | world model + planning (MPC) · TSDF fusion → mesh · Bayesian semantic mapping | — |
| LM · Language | a GPT from scratch (nanoGPT) · knowledge distillation | — |
| AG · Agents & RL | REINFORCE policy gradient · behavior cloning · agent + tool-use harness | — |
The twenty self-contained PyTorch labs are verified to train (each was run
to confirm loss drops / PSNR climbs / metrics beat chance); the three foundation
labs follow the official APIs and run on a Colab GPU. Every lab records its
checkpoint + loss/eval history + figures to a downloadable outputs/<lab>/, and an
optional cell publishes the run to the Hugging Face Hub (a model repo with a
metrics-and-plot model card) so you can gather them into a Collection. Open them from the dashboard's Training labs section, or see
notebooks/training/README.md for one-click Colab
badges. Set Runtime → T4 GPU first.
notebooks/advanced/ adds twenty-two heavy GPU pipelines on
real research repos for when you want the production tools, not a teaching toy:
| Track | Advanced pipelines |
|---|---|
| A · Human | MDM text-to-motion · 4D-Humans (HMR 2.0) mesh-from-video |
| B · 3D / rendering | 3D Gaussian Splatting (CUDA) · Nerfstudio nerfacto |
| C · Egocentric | VideoMAE fine-tune on EPIC/Ego4D · SAM 2 video segmentation · Whisper ASR fine-tune |
| D · Scene / world | SplaTAM (Gaussian SLAM) · DreamerV3 world model |
| LM · Language & multimodal | QLoRA · DPO · VLM fine-tune · Video-LM · RAG · LLM eval · Unsloth · RLHF (PPO) · Stable Diffusion LoRA · ControlNet · vLLM serving |
| AG · Agents & RL | LLM agent (tool use / ReAct) · Habitat embodied navigation |
The verified self-contained labs were each run to confirm real results; the advanced labs follow each project's official recipe and run on a Colab GPU (some need gated data), so they are flagged as not pre-executed. Every lab records its checkpoints / loss-eval history / outputs to a downloadable folder, every lab cross-links to its related tracks (chips + filters + an in-notebook note), and the Labs tab closes with a Future explore directions board — open cross-track projects that combine the labs.
These clone official repos, download multi-GB checkpoints/datasets, and require a
GPU — they're authored to each project's documented recipe and are not
pre-executed (expect to pin a version or two). See the dashboard's Advanced
labs · GPU section or notebooks/advanced/README.md.
Every lab publishes to the Hub as a model repo with a full card (base model · objective · dataset · config · evaluation · inference · limitations · failure cases · license · citation · reproducibility), gathered into one Collection. Try them all live in a single Space.
The 🚀 demos Space runs real models per task: a small instruct LLM (chat · next-action · ReAct tool use), a class-conditional DDPM that generates digits, a CartPole agent, and a world-model CEM planner — plus a gallery of every trained model's figure + metrics.

19 trained · 26 documented placeholders · 45 repos total — one click trains a placeholder into a real model.
A · Human modeling & motion
| Model | Status | Headline result | Links |
|---|---|---|---|
| 2D pose estimation (heatmap regression) | ✅ trained | PCK 0.396 | 🤗 · ▶ |
| 4D-Humans (HMR 2.0) — mesh from video | 🚧 placeholder | pending (GPU) | 🤗 · ▶ |
| 6D vs Euler rotation regression | ✅ trained | geodesic err 0.0126 | 🤗 · ▶ |
| MDM — text-to-motion | 🚧 placeholder | pending (GPU) | 🤗 · ▶ |
| Motion diffusion (DDPM) | ✅ trained | final loss 0.159 | 🤗 · ▶ |
| SMPLify body fit | ✅ trained | reproj err 6.3e-05 | 🤗 · ▶ |
B · 3D / 4D & neural rendering
| Model | Status | Headline result | Links |
|---|---|---|---|
| 2D Gaussian Splatting | ✅ trained | PSNR 32.5 | 🤗 · ▶ |
| 3D Gaussian Splatting | 🚧 placeholder | pending (GPU) | 🤗 · ▶ |
| ICP point-cloud registration | ✅ trained | RMSE 0.0124 | 🤗 · ▶ |
| Masked Autoencoder (MAE) | ✅ trained | test MSE 0.136 | 🤗 · ▶ |
| Multiresolution hash grid (Instant-NGP) | ✅ trained | PSNR 64.2 | 🤗 · ▶ |
| NeRF from scratch (tiny_nerf) | 🚧 placeholder | pending (GPU) | 🤗 · ▶ |
| Nerfstudio nerfacto | 🚧 placeholder | pending (GPU) | 🤗 · ▶ |
| Neural SDF (DeepSDF-style) | ✅ trained | L1 err 0.016 | 🤗 · ▶ |
C · Egocentric vision
| Model | Status | Headline result | Links |
|---|---|---|---|
| Action anticipation (LSTM) | ✅ trained | top-1 0.529 | 🤗 · ▶ |
| CLIP: zero-shot vs. probe | 🚧 placeholder | pending (GPU) | 🤗 · ▶ |
| DINOv2 features + probe | 🚧 placeholder | pending (GPU) | 🤗 · ▶ |
| Fine-tune VideoMAE | 🚧 placeholder | pending (GPU) | 🤗 · ▶ |
| SAM 2 — video segmentation | 🚧 placeholder | pending (GPU) | 🤗 · ▶ |
| SimCLR self-supervised pretraining | ✅ trained | probe acc 0.683 | 🤗 · ▶ |
| VideoMAE — egocentric fine-tune | 🚧 placeholder | pending (GPU) | 🤗 · ▶ |
| Whisper — fine-tune ASR | 🚧 placeholder | pending (GPU) | 🤗 · ▶ |
D · Scene & world models
| Model | Status | Headline result | Links |
|---|---|---|---|
| Bayesian semantic mapping | ✅ trained | map acc 1 | 🤗 · ▶ |
| DreamerV3 — world-model RL | 🚧 placeholder | pending (GPU) | 🤗 · ▶ |
| SplaTAM — Gaussian-Splatting SLAM | 🚧 placeholder | pending (GPU) | 🤗 · ▶ |
| TSDF fusion → mesh | ✅ trained | mesh verts 27362 | 🤗 · ▶ |
| World model + planning (CEM) | ✅ trained | dyn MSE 8.76e-08 | 🤗 · ▶ |
LM · Language & multimodal
| Model | Status | Headline result | Links |
|---|---|---|---|
| ControlNet — conditional diffusion | 🚧 placeholder | pending (GPU) | 🤗 · ▶ |
| DPO — align an LLM | 🚧 placeholder | pending (GPU) | 🤗 · ▶ |
| Evaluate an LLM (lm-eval-harness) | 🚧 placeholder | pending (GPU) | 🤗 · ▶ |
| Fine-tune a VLM (vision-language) | 🚧 placeholder | pending (GPU) | 🤗 · ▶ |
| Knowledge distillation | ✅ trained | distilled acc 0.969 | 🤗 · ▶ |
| QLoRA — fine-tune an LLM | 🚧 placeholder | pending (GPU) | 🤗 · ▶ |
| RAG — retrieval-augmented generation | 🚧 placeholder | pending (GPU) | 🤗 · ▶ |
| RLHF — PPO fine-tuning | 🚧 placeholder | pending (GPU) | 🤗 · ▶ |
| Serve an LLM (vLLM) | 🚧 placeholder | pending (GPU) | 🤗 · ▶ |
| Stable Diffusion — LoRA / DreamBooth | 🚧 placeholder | pending (GPU) | 🤗 · ▶ |
| Unsloth — fast LLM fine-tune | 🚧 placeholder | pending (GPU) | 🤗 · ▶ |
| Video-LM (Qwen2-VL) | 🚧 placeholder | pending (GPU) | 🤗 · ▶ |
| nanoGPT — Tiny Shakespeare | ✅ trained | val loss 1.81 | 🤗 · ▶ |
AG · Agents & RL
| Model | Status | Headline result | Links |
|---|---|---|---|
| Agent + tool-use harness | ✅ trained | success 1 | 🤗 · ▶ |
| Behavior cloning (imitation) | ✅ trained | rollout 1 | 🤗 · ▶ |
| Habitat — embodied navigation | 🚧 placeholder | pending (GPU) | 🤗 · ▶ |
| LLM agent — tool use (ReAct) | 🚧 placeholder | pending (GPU) | 🤗 · ▶ |
| REINFORCE / actor-critic (CartPole) | ✅ trained | greedy return 449 | 🤗 · ▶ |
npm install
npm run dev # http://localhost:5173
npm run build # production build → dist/
Pushing to main auto-builds and deploys to GitHub Pages via the included
GitHub Actions workflow (.github/workflows/deploy.yml) — that's what serves the
live site above. It's a static SPA, so dist/ can also be hosted on any static
host (Netlify, Vercel, Cloudflare Pages).
112 commits
Jupyter Notebook
58.5%
TypeScript
31.1%
JavaScript
7.6%
Python
2.5%
Ropedia Academy — interactive bilingual learning app for egocentric vision, 3D reconstruction, human modeling, and world models.
3
stars
112
commits
Jupyter Notebook
primary language
Jul 16, 2026
updated
🌐 Live site · 4 tracks · 36 bilingual lessons · live 3D demos · spaced repetition
An interactive, bilingual (中文 / English) course on embodied & spatial AI — four connected tracks covering human modeling & motion, 3D/4D reconstruction & neural rendering, egocentric vision & interaction, and scene reconstruction & world models. Read lessons, play with live interactive demos (including real-time 3D), run the code in Colab, self-test, and review with spaced repetition. Runs entirely in the browser — no account required.
Beyond the per-lesson snippets, notebooks/training/ holds
twenty-three real, multi-cell Colab notebooks you can actually train — split into
clear blocks (data · model · train · compare) so you can step through and watch
each stage:
| Track | From scratch (PyTorch) | Foundation model |
|---|---|---|
| A · Human | SMPLify body fit · motion diffusion (DDPM) · 2D pose (heatmap) · 6D vs Euler rotation | — |
| B · 3D / rendering | NeRF (tiny_nerf) · neural SDF · 2D Gaussian Splatting · hash grid (Instant-NGP) · ICP registration · MAE pretraining | — |
| C · Egocentric | action anticipation (LSTM) · SimCLR self-supervised pretraining | CLIP probe · fine-tune VideoMAE · DINOv2 features |
| D · Scene / world | world model + planning (MPC) · TSDF fusion → mesh · Bayesian semantic mapping | — |
| LM · Language | a GPT from scratch (nanoGPT) · knowledge distillation | — |
| AG · Agents & RL | REINFORCE policy gradient · behavior cloning · agent + tool-use harness | — |
The twenty self-contained PyTorch labs are verified to train (each was run
to confirm loss drops / PSNR climbs / metrics beat chance); the three foundation
labs follow the official APIs and run on a Colab GPU. Every lab records its
checkpoint + loss/eval history + figures to a downloadable outputs/<lab>/, and an
optional cell publishes the run to the Hugging Face Hub (a model repo with a
metrics-and-plot model card) so you can gather them into a Collection. Open them from the dashboard's Training labs section, or see
notebooks/training/README.md for one-click Colab
badges. Set Runtime → T4 GPU first.
notebooks/advanced/ adds twenty-two heavy GPU pipelines on
real research repos for when you want the production tools, not a teaching toy:
| Track | Advanced pipelines |
|---|---|
| A · Human | MDM text-to-motion · 4D-Humans (HMR 2.0) mesh-from-video |
| B · 3D / rendering | 3D Gaussian Splatting (CUDA) · Nerfstudio nerfacto |
| C · Egocentric | VideoMAE fine-tune on EPIC/Ego4D · SAM 2 video segmentation · Whisper ASR fine-tune |
| D · Scene / world | SplaTAM (Gaussian SLAM) · DreamerV3 world model |
| LM · Language & multimodal | QLoRA · DPO · VLM fine-tune · Video-LM · RAG · LLM eval · Unsloth · RLHF (PPO) · Stable Diffusion LoRA · ControlNet · vLLM serving |
| AG · Agents & RL | LLM agent (tool use / ReAct) · Habitat embodied navigation |
The verified self-contained labs were each run to confirm real results; the advanced labs follow each project's official recipe and run on a Colab GPU (some need gated data), so they are flagged as not pre-executed. Every lab records its checkpoints / loss-eval history / outputs to a downloadable folder, every lab cross-links to its related tracks (chips + filters + an in-notebook note), and the Labs tab closes with a Future explore directions board — open cross-track projects that combine the labs.
These clone official repos, download multi-GB checkpoints/datasets, and require a
GPU — they're authored to each project's documented recipe and are not
pre-executed (expect to pin a version or two). See the dashboard's Advanced
labs · GPU section or notebooks/advanced/README.md.
Every lab publishes to the Hub as a model repo with a full card (base model · objective · dataset · config · evaluation · inference · limitations · failure cases · license · citation · reproducibility), gathered into one Collection. Try them all live in a single Space.
The 🚀 demos Space runs real models per task: a small instruct LLM (chat · next-action · ReAct tool use), a class-conditional DDPM that generates digits, a CartPole agent, and a world-model CEM planner — plus a gallery of every trained model's figure + metrics.

19 trained · 26 documented placeholders · 45 repos total — one click trains a placeholder into a real model.
A · Human modeling & motion
| Model | Status | Headline result | Links |
|---|---|---|---|
| 2D pose estimation (heatmap regression) | ✅ trained | PCK 0.396 | 🤗 · ▶ |
| 4D-Humans (HMR 2.0) — mesh from video | 🚧 placeholder | pending (GPU) | 🤗 · ▶ |
| 6D vs Euler rotation regression | ✅ trained | geodesic err 0.0126 | 🤗 · ▶ |
| MDM — text-to-motion | 🚧 placeholder | pending (GPU) | 🤗 · ▶ |
| Motion diffusion (DDPM) | ✅ trained | final loss 0.159 | 🤗 · ▶ |
| SMPLify body fit | ✅ trained | reproj err 6.3e-05 | 🤗 · ▶ |
B · 3D / 4D & neural rendering
| Model | Status | Headline result | Links |
|---|---|---|---|
| 2D Gaussian Splatting | ✅ trained | PSNR 32.5 | 🤗 · ▶ |
| 3D Gaussian Splatting | 🚧 placeholder | pending (GPU) | 🤗 · ▶ |
| ICP point-cloud registration | ✅ trained | RMSE 0.0124 | 🤗 · ▶ |
| Masked Autoencoder (MAE) | ✅ trained | test MSE 0.136 | 🤗 · ▶ |
| Multiresolution hash grid (Instant-NGP) | ✅ trained | PSNR 64.2 | 🤗 · ▶ |
| NeRF from scratch (tiny_nerf) | 🚧 placeholder | pending (GPU) | 🤗 · ▶ |
| Nerfstudio nerfacto | 🚧 placeholder | pending (GPU) | 🤗 · ▶ |
| Neural SDF (DeepSDF-style) | ✅ trained | L1 err 0.016 | 🤗 · ▶ |
C · Egocentric vision
| Model | Status | Headline result | Links |
|---|---|---|---|
| Action anticipation (LSTM) | ✅ trained | top-1 0.529 | 🤗 · ▶ |
| CLIP: zero-shot vs. probe | 🚧 placeholder | pending (GPU) | 🤗 · ▶ |
| DINOv2 features + probe | 🚧 placeholder | pending (GPU) | 🤗 · ▶ |
| Fine-tune VideoMAE | 🚧 placeholder | pending (GPU) | 🤗 · ▶ |
| SAM 2 — video segmentation | 🚧 placeholder | pending (GPU) | 🤗 · ▶ |
| SimCLR self-supervised pretraining | ✅ trained | probe acc 0.683 | 🤗 · ▶ |
| VideoMAE — egocentric fine-tune | 🚧 placeholder | pending (GPU) | 🤗 · ▶ |
| Whisper — fine-tune ASR | 🚧 placeholder | pending (GPU) | 🤗 · ▶ |
D · Scene & world models
| Model | Status | Headline result | Links |
|---|---|---|---|
| Bayesian semantic mapping | ✅ trained | map acc 1 | 🤗 · ▶ |
| DreamerV3 — world-model RL | 🚧 placeholder | pending (GPU) | 🤗 · ▶ |
| SplaTAM — Gaussian-Splatting SLAM | 🚧 placeholder | pending (GPU) | 🤗 · ▶ |
| TSDF fusion → mesh | ✅ trained | mesh verts 27362 | 🤗 · ▶ |
| World model + planning (CEM) | ✅ trained | dyn MSE 8.76e-08 | 🤗 · ▶ |
LM · Language & multimodal
| Model | Status | Headline result | Links |
|---|---|---|---|
| ControlNet — conditional diffusion | 🚧 placeholder | pending (GPU) | 🤗 · ▶ |
| DPO — align an LLM | 🚧 placeholder | pending (GPU) | 🤗 · ▶ |
| Evaluate an LLM (lm-eval-harness) | 🚧 placeholder | pending (GPU) | 🤗 · ▶ |
| Fine-tune a VLM (vision-language) | 🚧 placeholder | pending (GPU) | 🤗 · ▶ |
| Knowledge distillation | ✅ trained | distilled acc 0.969 | 🤗 · ▶ |
| QLoRA — fine-tune an LLM | 🚧 placeholder | pending (GPU) | 🤗 · ▶ |
| RAG — retrieval-augmented generation | 🚧 placeholder | pending (GPU) | 🤗 · ▶ |
| RLHF — PPO fine-tuning | 🚧 placeholder | pending (GPU) | 🤗 · ▶ |
| Serve an LLM (vLLM) | 🚧 placeholder | pending (GPU) | 🤗 · ▶ |
| Stable Diffusion — LoRA / DreamBooth | 🚧 placeholder | pending (GPU) | 🤗 · ▶ |
| Unsloth — fast LLM fine-tune | 🚧 placeholder | pending (GPU) | 🤗 · ▶ |
| Video-LM (Qwen2-VL) | 🚧 placeholder | pending (GPU) | 🤗 · ▶ |
| nanoGPT — Tiny Shakespeare | ✅ trained | val loss 1.81 | 🤗 · ▶ |
AG · Agents & RL
| Model | Status | Headline result | Links |
|---|---|---|---|
| Agent + tool-use harness | ✅ trained | success 1 | 🤗 · ▶ |
| Behavior cloning (imitation) | ✅ trained | rollout 1 | 🤗 · ▶ |
| Habitat — embodied navigation | 🚧 placeholder | pending (GPU) | 🤗 · ▶ |
| LLM agent — tool use (ReAct) | 🚧 placeholder | pending (GPU) | 🤗 · ▶ |
| REINFORCE / actor-critic (CartPole) | ✅ trained | greedy return 449 | 🤗 · ▶ |
npm install
npm run dev # http://localhost:5173
npm run build # production build → dist/
Pushing to main auto-builds and deploys to GitHub Pages via the included
GitHub Actions workflow (.github/workflows/deploy.yml) — that's what serves the
live site above. It's a static SPA, so dist/ can also be hosted on any static
host (Netlify, Vercel, Cloudflare Pages).
112 commits
Jupyter Notebook
58.5%
TypeScript
31.1%
JavaScript
7.6%
Python
2.5%