Learn World Models（⚠️ Alpha Preview）

Learn world models by building them: from the intuition behind latent dynamics to a working simulation, planning, and evaluation system.

📖 Read the course online →

[!CAUTION] ⚠️ Alpha Preview: This is an early build. Content is still being completed and revised: sections, examples, and wording may continue to change. Feedback via Issues is welcome.

✨ Preview

🏠 Course Home

Structured learning path with lecture and project cards.

Course home

📖 Lecture Pages

Concept-first explanations with mermaid diagrams and background callouts for deep-learning readers.

Lecture page

🗂️ Architecture Deep Dive

Nine architecture families, three planning mechanisms, and side-by-side comparison tables.

Architecture lecture

What this course covers

Five lectures and six projects that take you from the intuition behind world models to training, evaluating, and causally probing modern world-model systems.

#	Type	Title	Core Topics
L01	Lecture	World Model Definition & Origins	Working interfaces, rendering/simulation/planning, Craik, predictive coding, capability taxonomy
L02	Lecture	State Estimation, Encoding & Dynamics	Partial observability, belief state, VAE, GRU → MDN-RNN → RSSM, free rollout
L03	Lecture	Architecture Patterns, Learning Paradigms & Planning	Planning and control, backbone selection, nine architecture families, optional frontier survey
L04	Lecture	Diagnosing World Models	Representation, dynamics, rollout, task-signal, planning, and deployment diagnostics
L05	Lecture	Frontier Debates	Language vs physical grounding, Bitter Lesson, AGI as a research target
P01	Project	Train a VAE Encoder	Small CNN VAE on 64×64 pixels. ELBO loss curve. Latent slider visualization
P02	Project	Build an RSSM Dynamics Model	GRU, MDN-RNN, and RSSM compared. Prior vs posterior rollout plots
P03	Project	Train a Dreamer Agent	Full training loop: encoder + RSSM + latent Actor-Critic on a small pixel env
P04	Project	Swap the Dynamics Backbone	Replace RSSM with a small causal Transformer (STORM-style). Architecture comparison
P05	Project	World Model Evaluation Dashboard	Per-model metrics side by side: FID, reward correlation, PSNR, latent drift
P06	Project	Counterfactual Action-Conditioned World Model	Interventional and counterfactual rollouts, inverse-dynamics regularization, action-influence metric

Curriculum flow

Stage	Read	Then practice
Foundations	L01	Build the shared interface vocabulary before history and taxonomy
State estimation	L02: Observation, State, and Belief	Connect partial observations and history to latent state
Representation	L02: Observation Encoding	P01: Train a VAE Encoder
Dynamics and rollout reliability	L02: Latent Dynamics, then Training Distributions and Free Rollouts	P02: Build and diagnose an RSSM Dynamics Model
Control	L03: Planning and Control	P03: Train a Dreamer Agent
Backbone choice	L03: Backbone Selection	P04: Swap the Dynamics Backbone
Research orientation	L03: Optional Frontier Survey	Optional reading, no project prerequisite
Diagnosis	L04: Diagnosing World Models	P05: Evaluation Dashboard and P06: Counterfactual Fidelity
Open questions	L05	Synthesize the unresolved debates

Suggested path: L01, L02 Observation/State/Belief, L02 Observation Encoding, P01, L02 Latent Dynamics, L02 Training Distributions and Free Rollouts, P02, L03 Planning and Control, P03, L03 Backbone Selection, P04, optional L03 frontier survey, L04, P05, P06, L05.

You do not need to finish all theory before starting a project. Build, then come back with questions.

Quick start

npm install
npm run docs:dev        # dev server with hot reload
npm run docs:build      # production build
npm run docs:preview    # preview built site

To refresh the README screenshots after a build:

npm run docs:build
npm run screenshots:readme

Repo structure

learn-world-model/
├── docs/                                  # VitePress documentation site
│   ├── .vitepress/config.mts             # nav and sidebar (EN + ZH + KO)
│   ├── en/lectures/                       # 5 English lecture modules
│   ├── zh/lectures/                       # 5 Chinese lecture modules
│   ├── ko/lectures/                       # 5 Korean lecture modules
│   ├── en/projects/                       # 6 English project pages
│   ├── zh/projects/                       # 6 Chinese project pages
│   └── ko/projects/                       # 6 Korean project pages
├── external/world-model-tutorial/         # PyTorch source referenced by projects
│   └── references.md                      # four-era history and architecture survey
├── scripts/                               # build utilities (screenshots, PDF)
└── package.json

Community

Scan the QR code to join the WeChat discussion group (微信交流群):

Contributing

Contributions are welcome. Before submitting a pull request, read CLAUDE.md for the writing style rules that apply to all lecture and project files (no em dashes, no linear mermaid diagrams, no arrow-chain prose, EN/ZH sync, and others). Content that does not follow those rules will be asked to revise before merging.

Contributors

Name	Role	Affiliation	GitHub
Zhimin Zhao	Project Lead	Queen's University	@zhimin-z
Qi Wang	Project Lead	Chinese Academy of Sciences	@qiwang067
Dongwoo Ro	Contributor		@dwro0121
Xun Wang	Contributor		@wangxunx