Learn World Models（⚠️ Alpha Preview）

Learn world models by building them: from the intuition behind latent dynamics to a working simulation, planning, and evaluation system.

[!CAUTION] ⚠️ Alpha Preview: This is an early build. Content is still being completed and revised: sections, examples, and wording may continue to change. Feedback via Issues is welcome.

✨ Preview

🏠 Course Home

Structured learning path with lecture and project cards.

Course home

📖 Lecture Pages

Concept-first explanations with mermaid diagrams and background callouts for deep-learning readers.

Lecture page

🗂️ Architecture Deep Dive

Seven architecture families, three planning mechanisms, side-by-side comparison tables.

Architecture lecture

What this course covers

Five lectures and five projects that take you from the intuition behind world models to a working three-model evaluation dashboard.

#	Type	Title	Core Topics
L01	Lecture	Internal Simulation & Historical Context	Craik's mental models, predictive coding, four eras of world model evolution
L02	Lecture	Observation Encoding & Latent Dynamics	VAE, CNN encoder, ELBO, GRU → MDN-RNN → RSSM
L03	Lecture	Architecture Patterns, Learning Paradigms & Planning	Seven architecture families, CEM-MPC, latent Actor-Critic, TD-MPC
L04	Lecture	Evaluation by World Model	FID, reward correlation, consistency loss, PSNR, horizon drift
L05	Lecture	Frontier Debates	Language vs physical grounding, Bitter Lesson, AGI as a research target
P01	Project	Train a VAE Encoder	Small CNN VAE on 64×64 pixels; ELBO loss curve; latent slider visualization
P02	Project	Build an RSSM Dynamics Model	GRU, MDN-RNN, and RSSM compared; prior vs posterior rollout plots
P03	Project	Train a Dreamer Agent	Full training loop: encoder + RSSM + latent Actor-Critic on a small pixel env
P04	Project	Swap the Dynamics Backbone	Replace RSSM with a small causal Transformer (STORM-style); architecture comparison
P05	Project	World Model Evaluation Dashboard	Per-model metrics side by side: FID, reward correlation, PSNR, latent drift

Curriculum flow

flowchart TD
    L01["L01 History and Intuition"] --> L02A
    L02A["L02 Part A: VAE Encoder"] --> P01["P01 Train VAE, visualize latent space"]
    L02A --> L02B["L02 Part B: GRU to RSSM"]
    L02B --> P02["P02 Build RSSM, compare prior vs posterior"]
    L02B --> L03A["L03 Part A: Architecture Patterns"]
    L03A --> L03B["L03 Part B: Planning mechanisms"]
    L03B --> P03["P03 Train Dreamer agent"]
    P02 --> P04["P04 Swap RSSM for Transformer backbone"]
    L03A --> P04
    P03 & P04 --> L04["L04 Evaluation metrics"]
    L04 --> P05["P05 Evaluation dashboard"]
    P05 --> L05["L05 Frontier Debates"]

Suggested path: L01, L02, P01, P02, L03, P03, P04, L04, P05, L05

You do not need to finish all theory before starting a project. Build, then come back with questions.

Quick start

npm install
npm run docs:dev        # dev server with hot reload
npm run docs:build      # production build
npm run docs:preview    # preview built site

To refresh the README screenshots after a build:

npm run docs:build
npm run screenshots:readme

Repo structure

learn-world-model/
├── docs/                                  # VitePress documentation site
│   ├── .vitepress/config.mts             # nav and sidebar (EN + ZH)
│   ├── en/lectures/                       # 5 English lecture pages
│   ├── zh/lectures/                       # 5 Chinese lecture pages
│   ├── en/projects/                       # 5 English project pages
│   └── zh/projects/                       # 5 Chinese project pages
├── external/world-model-tutorial/         # PyTorch source referenced by projects
│   └── references.md                      # four-era history and architecture survey
├── scripts/                               # build utilities (screenshots, PDF)
└── package.json

Community

Scan the QR code to join the WeChat discussion group (微信交流群):

Contributing

Contributions are welcome. Before submitting a pull request, read CLAUDE.md for the writing style rules that apply to all lecture and project files (no em dashes, no linear mermaid diagrams, no arrow-chain prose, EN/ZH sync, and others). Content that does not follow those rules will be asked to revise before merging.

Contributors (Tutorial)

Name	Role	Affiliation	GitHub
Zhimin Zhao	Project Lead	Queen's University	@zhimin-z
Qi Wang	Project Lead	Shanghai Jiao Tong University	@qiwang067