ChaoYue0307/ropedia-academy

Ropedia Academy — interactive bilingual learning app for egocentric vision, 3D reconstruction, human modeling, and world models.

3

stars

112

commits

Jupyter Notebook

primary language

Jul 16, 2026

updated

chaoyue0307.github.io/ropedia-academy/

README

Ropedia Academy — learn embodied spatial AI: egocentric vision, 3D reconstruction, human motion, world models

Ropedia Academy

🌐 Live site · 4 tracks · 36 bilingual lessons · live 3D demos · spaced repetition

An interactive, bilingual (中文 / English) course on embodied & spatial AI — four connected tracks covering human modeling & motion, 3D/4D reconstruction & neural rendering, egocentric vision & interaction, and scene reconstruction & world models. Read lessons, play with live interactive demos (including real-time 3D), run the code in Colab, self-test, and review with spaced repetition. Runs entirely in the browser — no account required.

tracks lessons mode-中%2FEN-blue

Features

  • 4 tracks · 36 lessons, each with a bilingual explanation, key terms, key papers, external links, cross-track links, and self-check questions.
  • An interactive demo in every lesson — real-time three.js 3D (Gaussian splatting, a raymarched NeRF volume, an articulated SMPL body) plus explorable diagrams (triangulation, bundle adjustment, rotation continuity, hash grids, TSDF fusion, SLAM loop closure, reference frames, world-model rollouts, …).
  • Math & code — KaTeX formulas and a runnable Python/PyTorch snippet per lesson, each with one-click Open in Colab; its real output (printed values
    • figure) shows inline behind a predict-it, then reveal toggle.
  • Bilingual reading (中文 / English / 双语) and hover-to-define glossary tooltips on foundational terms.
  • Self-graded checks & quiz mode, spaced repetition (SM-2) with a 7-day forecast, a cross-track concept map, progress tracking, and a ⌘K palette.
  • Light / dark theme, mobile-friendly, local-first (no account required).

Training labs

Beyond the per-lesson snippets, notebooks/training/ holds twenty-three real, multi-cell Colab notebooks you can actually train — split into clear blocks (data · model · train · compare) so you can step through and watch each stage:

TrackFrom scratch (PyTorch)Foundation model
A · HumanSMPLify body fit · motion diffusion (DDPM) · 2D pose (heatmap) · 6D vs Euler rotation
B · 3D / renderingNeRF (tiny_nerf) · neural SDF · 2D Gaussian Splatting · hash grid (Instant-NGP) · ICP registration · MAE pretraining
C · Egocentricaction anticipation (LSTM) · SimCLR self-supervised pretrainingCLIP probe · fine-tune VideoMAE · DINOv2 features
D · Scene / worldworld model + planning (MPC) · TSDF fusion → mesh · Bayesian semantic mapping
LM · Languagea GPT from scratch (nanoGPT) · knowledge distillation
AG · Agents & RLREINFORCE policy gradient · behavior cloning · agent + tool-use harness

The twenty self-contained PyTorch labs are verified to train (each was run to confirm loss drops / PSNR climbs / metrics beat chance); the three foundation labs follow the official APIs and run on a Colab GPU. Every lab records its checkpoint + loss/eval history + figures to a downloadable outputs/<lab>/, and an optional cell publishes the run to the Hugging Face Hub (a model repo with a metrics-and-plot model card) so you can gather them into a Collection. Open them from the dashboard's Training labs section, or see notebooks/training/README.md for one-click Colab badges. Set Runtime → T4 GPU first.

Advanced labs (heavy · real repos · GPU)

notebooks/advanced/ adds twenty-two heavy GPU pipelines on real research repos for when you want the production tools, not a teaching toy:

TrackAdvanced pipelines
A · HumanMDM text-to-motion · 4D-Humans (HMR 2.0) mesh-from-video
B · 3D / rendering3D Gaussian Splatting (CUDA) · Nerfstudio nerfacto
C · EgocentricVideoMAE fine-tune on EPIC/Ego4D · SAM 2 video segmentation · Whisper ASR fine-tune
D · Scene / worldSplaTAM (Gaussian SLAM) · DreamerV3 world model
LM · Language & multimodalQLoRA · DPO · VLM fine-tune · Video-LM · RAG · LLM eval · Unsloth · RLHF (PPO) · Stable Diffusion LoRA · ControlNet · vLLM serving
AG · Agents & RLLLM agent (tool use / ReAct) · Habitat embodied navigation

The verified self-contained labs were each run to confirm real results; the advanced labs follow each project's official recipe and run on a Colab GPU (some need gated data), so they are flagged as not pre-executed. Every lab records its checkpoints / loss-eval history / outputs to a downloadable folder, every lab cross-links to its related tracks (chips + filters + an in-notebook note), and the Labs tab closes with a Future explore directions board — open cross-track projects that combine the labs.

These clone official repos, download multi-GB checkpoints/datasets, and require a GPU — they're authored to each project's documented recipe and are not pre-executed (expect to pin a version or two). See the dashboard's Advanced labs · GPU section or notebooks/advanced/README.md.

Models on Hugging Face

Every lab publishes to the Hub as a model repo with a full card (base model · objective · dataset · config · evaluation · inference · limitations · failure cases · license · citation · reproducibility), gathered into one Collection. Try them all live in a single Space.

Live demos Models

The 🚀 demos Space runs real models per task: a small instruct LLM (chat · next-action · ReAct tool use), a class-conditional DDPM that generates digits, a CartPole agent, and a world-model CEM planner — plus a gallery of every trained model's figure + metrics.

Real outputs from Ropedia Academy models: 2D Gaussian Splatting on a photo, MAE reconstruction, REINFORCE solving CartPole, knowledge distillation, SimCLR multi-seed robustness, world-model CEM planning

19 trained · 26 documented placeholders · 45 repos total — one click trains a placeholder into a real model.

A · Human modeling & motion

ModelStatusHeadline resultLinks
2D pose estimation (heatmap regression)✅ trainedPCK 0.396🤗 ·
4D-Humans (HMR 2.0) — mesh from video🚧 placeholderpending (GPU)🤗 ·
6D vs Euler rotation regression✅ trainedgeodesic err 0.0126🤗 ·
MDM — text-to-motion🚧 placeholderpending (GPU)🤗 ·
Motion diffusion (DDPM)✅ trainedfinal loss 0.159🤗 ·
SMPLify body fit✅ trainedreproj err 6.3e-05🤗 ·

B · 3D / 4D & neural rendering

ModelStatusHeadline resultLinks
2D Gaussian Splatting✅ trainedPSNR 32.5🤗 ·
3D Gaussian Splatting🚧 placeholderpending (GPU)🤗 ·
ICP point-cloud registration✅ trainedRMSE 0.0124🤗 ·
Masked Autoencoder (MAE)✅ trainedtest MSE 0.136🤗 ·
Multiresolution hash grid (Instant-NGP)✅ trainedPSNR 64.2🤗 ·
NeRF from scratch (tiny_nerf)🚧 placeholderpending (GPU)🤗 ·
Nerfstudio nerfacto🚧 placeholderpending (GPU)🤗 ·
Neural SDF (DeepSDF-style)✅ trainedL1 err 0.016🤗 ·

C · Egocentric vision

ModelStatusHeadline resultLinks
Action anticipation (LSTM)✅ trainedtop-1 0.529🤗 ·
CLIP: zero-shot vs. probe🚧 placeholderpending (GPU)🤗 ·
DINOv2 features + probe🚧 placeholderpending (GPU)🤗 ·
Fine-tune VideoMAE🚧 placeholderpending (GPU)🤗 ·
SAM 2 — video segmentation🚧 placeholderpending (GPU)🤗 ·
SimCLR self-supervised pretraining✅ trainedprobe acc 0.683🤗 ·
VideoMAE — egocentric fine-tune🚧 placeholderpending (GPU)🤗 ·
Whisper — fine-tune ASR🚧 placeholderpending (GPU)🤗 ·

D · Scene & world models

ModelStatusHeadline resultLinks
Bayesian semantic mapping✅ trainedmap acc 1🤗 ·
DreamerV3 — world-model RL🚧 placeholderpending (GPU)🤗 ·
SplaTAM — Gaussian-Splatting SLAM🚧 placeholderpending (GPU)🤗 ·
TSDF fusion → mesh✅ trainedmesh verts 27362🤗 ·
World model + planning (CEM)✅ traineddyn MSE 8.76e-08🤗 ·

LM · Language & multimodal

ModelStatusHeadline resultLinks
ControlNet — conditional diffusion🚧 placeholderpending (GPU)🤗 ·
DPO — align an LLM🚧 placeholderpending (GPU)🤗 ·
Evaluate an LLM (lm-eval-harness)🚧 placeholderpending (GPU)🤗 ·
Fine-tune a VLM (vision-language)🚧 placeholderpending (GPU)🤗 ·
Knowledge distillation✅ traineddistilled acc 0.969🤗 ·
QLoRA — fine-tune an LLM🚧 placeholderpending (GPU)🤗 ·
RAG — retrieval-augmented generation🚧 placeholderpending (GPU)🤗 ·
RLHF — PPO fine-tuning🚧 placeholderpending (GPU)🤗 ·
Serve an LLM (vLLM)🚧 placeholderpending (GPU)🤗 ·
Stable Diffusion — LoRA / DreamBooth🚧 placeholderpending (GPU)🤗 ·
Unsloth — fast LLM fine-tune🚧 placeholderpending (GPU)🤗 ·
Video-LM (Qwen2-VL)🚧 placeholderpending (GPU)🤗 ·
nanoGPT — Tiny Shakespeare✅ trainedval loss 1.81🤗 ·

AG · Agents & RL

ModelStatusHeadline resultLinks
Agent + tool-use harness✅ trainedsuccess 1🤗 ·
Behavior cloning (imitation)✅ trainedrollout 1🤗 ·
Habitat — embodied navigation🚧 placeholderpending (GPU)🤗 ·
LLM agent — tool use (ReAct)🚧 placeholderpending (GPU)🤗 ·
REINFORCE / actor-critic (CartPole)✅ trainedgreedy return 449🤗 ·

Run & deploy

npm install
npm run dev      # http://localhost:5173
npm run build    # production build → dist/

Pushing to main auto-builds and deploys to GitHub Pages via the included GitHub Actions workflow (.github/workflows/deploy.yml) — that's what serves the live site above. It's a static SPA, so dist/ can also be hosted on any static host (Netlify, Vercel, Cloudflare Pages).

Contributors

ChaoYue0307

112 commits

ChaoYue0307/ropedia-academy

Ropedia Academy — interactive bilingual learning app for egocentric vision, 3D reconstruction, human modeling, and world models.

3

stars

112

commits

Jupyter Notebook

primary language

Jul 16, 2026

updated

chaoyue0307.github.io/ropedia-academy/

README

Ropedia Academy — learn embodied spatial AI: egocentric vision, 3D reconstruction, human motion, world models

Ropedia Academy

🌐 Live site · 4 tracks · 36 bilingual lessons · live 3D demos · spaced repetition

An interactive, bilingual (中文 / English) course on embodied & spatial AI — four connected tracks covering human modeling & motion, 3D/4D reconstruction & neural rendering, egocentric vision & interaction, and scene reconstruction & world models. Read lessons, play with live interactive demos (including real-time 3D), run the code in Colab, self-test, and review with spaced repetition. Runs entirely in the browser — no account required.

tracks lessons mode-中%2FEN-blue

Features

  • 4 tracks · 36 lessons, each with a bilingual explanation, key terms, key papers, external links, cross-track links, and self-check questions.
  • An interactive demo in every lesson — real-time three.js 3D (Gaussian splatting, a raymarched NeRF volume, an articulated SMPL body) plus explorable diagrams (triangulation, bundle adjustment, rotation continuity, hash grids, TSDF fusion, SLAM loop closure, reference frames, world-model rollouts, …).
  • Math & code — KaTeX formulas and a runnable Python/PyTorch snippet per lesson, each with one-click Open in Colab; its real output (printed values
    • figure) shows inline behind a predict-it, then reveal toggle.
  • Bilingual reading (中文 / English / 双语) and hover-to-define glossary tooltips on foundational terms.
  • Self-graded checks & quiz mode, spaced repetition (SM-2) with a 7-day forecast, a cross-track concept map, progress tracking, and a ⌘K palette.
  • Light / dark theme, mobile-friendly, local-first (no account required).

Training labs

Beyond the per-lesson snippets, notebooks/training/ holds twenty-three real, multi-cell Colab notebooks you can actually train — split into clear blocks (data · model · train · compare) so you can step through and watch each stage:

TrackFrom scratch (PyTorch)Foundation model
A · HumanSMPLify body fit · motion diffusion (DDPM) · 2D pose (heatmap) · 6D vs Euler rotation
B · 3D / renderingNeRF (tiny_nerf) · neural SDF · 2D Gaussian Splatting · hash grid (Instant-NGP) · ICP registration · MAE pretraining
C · Egocentricaction anticipation (LSTM) · SimCLR self-supervised pretrainingCLIP probe · fine-tune VideoMAE · DINOv2 features
D · Scene / worldworld model + planning (MPC) · TSDF fusion → mesh · Bayesian semantic mapping
LM · Languagea GPT from scratch (nanoGPT) · knowledge distillation
AG · Agents & RLREINFORCE policy gradient · behavior cloning · agent + tool-use harness

The twenty self-contained PyTorch labs are verified to train (each was run to confirm loss drops / PSNR climbs / metrics beat chance); the three foundation labs follow the official APIs and run on a Colab GPU. Every lab records its checkpoint + loss/eval history + figures to a downloadable outputs/<lab>/, and an optional cell publishes the run to the Hugging Face Hub (a model repo with a metrics-and-plot model card) so you can gather them into a Collection. Open them from the dashboard's Training labs section, or see notebooks/training/README.md for one-click Colab badges. Set Runtime → T4 GPU first.

Advanced labs (heavy · real repos · GPU)

notebooks/advanced/ adds twenty-two heavy GPU pipelines on real research repos for when you want the production tools, not a teaching toy:

TrackAdvanced pipelines
A · HumanMDM text-to-motion · 4D-Humans (HMR 2.0) mesh-from-video
B · 3D / rendering3D Gaussian Splatting (CUDA) · Nerfstudio nerfacto
C · EgocentricVideoMAE fine-tune on EPIC/Ego4D · SAM 2 video segmentation · Whisper ASR fine-tune
D · Scene / worldSplaTAM (Gaussian SLAM) · DreamerV3 world model
LM · Language & multimodalQLoRA · DPO · VLM fine-tune · Video-LM · RAG · LLM eval · Unsloth · RLHF (PPO) · Stable Diffusion LoRA · ControlNet · vLLM serving
AG · Agents & RLLLM agent (tool use / ReAct) · Habitat embodied navigation

The verified self-contained labs were each run to confirm real results; the advanced labs follow each project's official recipe and run on a Colab GPU (some need gated data), so they are flagged as not pre-executed. Every lab records its checkpoints / loss-eval history / outputs to a downloadable folder, every lab cross-links to its related tracks (chips + filters + an in-notebook note), and the Labs tab closes with a Future explore directions board — open cross-track projects that combine the labs.

These clone official repos, download multi-GB checkpoints/datasets, and require a GPU — they're authored to each project's documented recipe and are not pre-executed (expect to pin a version or two). See the dashboard's Advanced labs · GPU section or notebooks/advanced/README.md.

Models on Hugging Face

Every lab publishes to the Hub as a model repo with a full card (base model · objective · dataset · config · evaluation · inference · limitations · failure cases · license · citation · reproducibility), gathered into one Collection. Try them all live in a single Space.

Live demos Models

The 🚀 demos Space runs real models per task: a small instruct LLM (chat · next-action · ReAct tool use), a class-conditional DDPM that generates digits, a CartPole agent, and a world-model CEM planner — plus a gallery of every trained model's figure + metrics.

Real outputs from Ropedia Academy models: 2D Gaussian Splatting on a photo, MAE reconstruction, REINFORCE solving CartPole, knowledge distillation, SimCLR multi-seed robustness, world-model CEM planning

19 trained · 26 documented placeholders · 45 repos total — one click trains a placeholder into a real model.

A · Human modeling & motion

ModelStatusHeadline resultLinks
2D pose estimation (heatmap regression)✅ trainedPCK 0.396🤗 ·
4D-Humans (HMR 2.0) — mesh from video🚧 placeholderpending (GPU)🤗 ·
6D vs Euler rotation regression✅ trainedgeodesic err 0.0126🤗 ·
MDM — text-to-motion🚧 placeholderpending (GPU)🤗 ·
Motion diffusion (DDPM)✅ trainedfinal loss 0.159🤗 ·
SMPLify body fit✅ trainedreproj err 6.3e-05🤗 ·

B · 3D / 4D & neural rendering

ModelStatusHeadline resultLinks
2D Gaussian Splatting✅ trainedPSNR 32.5🤗 ·
3D Gaussian Splatting🚧 placeholderpending (GPU)🤗 ·
ICP point-cloud registration✅ trainedRMSE 0.0124🤗 ·
Masked Autoencoder (MAE)✅ trainedtest MSE 0.136🤗 ·
Multiresolution hash grid (Instant-NGP)✅ trainedPSNR 64.2🤗 ·
NeRF from scratch (tiny_nerf)🚧 placeholderpending (GPU)🤗 ·
Nerfstudio nerfacto🚧 placeholderpending (GPU)🤗 ·
Neural SDF (DeepSDF-style)✅ trainedL1 err 0.016🤗 ·

C · Egocentric vision

ModelStatusHeadline resultLinks
Action anticipation (LSTM)✅ trainedtop-1 0.529🤗 ·
CLIP: zero-shot vs. probe🚧 placeholderpending (GPU)🤗 ·
DINOv2 features + probe🚧 placeholderpending (GPU)🤗 ·
Fine-tune VideoMAE🚧 placeholderpending (GPU)🤗 ·
SAM 2 — video segmentation🚧 placeholderpending (GPU)🤗 ·
SimCLR self-supervised pretraining✅ trainedprobe acc 0.683🤗 ·
VideoMAE — egocentric fine-tune🚧 placeholderpending (GPU)🤗 ·
Whisper — fine-tune ASR🚧 placeholderpending (GPU)🤗 ·

D · Scene & world models

ModelStatusHeadline resultLinks
Bayesian semantic mapping✅ trainedmap acc 1🤗 ·
DreamerV3 — world-model RL🚧 placeholderpending (GPU)🤗 ·
SplaTAM — Gaussian-Splatting SLAM🚧 placeholderpending (GPU)🤗 ·
TSDF fusion → mesh✅ trainedmesh verts 27362🤗 ·
World model + planning (CEM)✅ traineddyn MSE 8.76e-08🤗 ·

LM · Language & multimodal

ModelStatusHeadline resultLinks
ControlNet — conditional diffusion🚧 placeholderpending (GPU)🤗 ·
DPO — align an LLM🚧 placeholderpending (GPU)🤗 ·
Evaluate an LLM (lm-eval-harness)🚧 placeholderpending (GPU)🤗 ·
Fine-tune a VLM (vision-language)🚧 placeholderpending (GPU)🤗 ·
Knowledge distillation✅ traineddistilled acc 0.969🤗 ·
QLoRA — fine-tune an LLM🚧 placeholderpending (GPU)🤗 ·
RAG — retrieval-augmented generation🚧 placeholderpending (GPU)🤗 ·
RLHF — PPO fine-tuning🚧 placeholderpending (GPU)🤗 ·
Serve an LLM (vLLM)🚧 placeholderpending (GPU)🤗 ·
Stable Diffusion — LoRA / DreamBooth🚧 placeholderpending (GPU)🤗 ·
Unsloth — fast LLM fine-tune🚧 placeholderpending (GPU)🤗 ·
Video-LM (Qwen2-VL)🚧 placeholderpending (GPU)🤗 ·
nanoGPT — Tiny Shakespeare✅ trainedval loss 1.81🤗 ·

AG · Agents & RL

ModelStatusHeadline resultLinks
Agent + tool-use harness✅ trainedsuccess 1🤗 ·
Behavior cloning (imitation)✅ trainedrollout 1🤗 ·
Habitat — embodied navigation🚧 placeholderpending (GPU)🤗 ·
LLM agent — tool use (ReAct)🚧 placeholderpending (GPU)🤗 ·
REINFORCE / actor-critic (CartPole)✅ trainedgreedy return 449🤗 ·

Run & deploy

npm install
npm run dev      # http://localhost:5173
npm run build    # production build → dist/

Pushing to main auto-builds and deploys to GitHub Pages via the included GitHub Actions workflow (.github/workflows/deploy.yml) — that's what serves the live site above. It's a static SPA, so dist/ can also be hosted on any static host (Netlify, Vercel, Cloudflare Pages).

Contributors

ChaoYue0307

112 commits

Languages

Jupyter Notebook

58.5%

TypeScript

31.1%

JavaScript

7.6%

Python

2.5%