{
"cells": [
{
"cell_type": "markdown",
"id": "0d9dfa25",
"metadata": {},
"source": [
"# 03｜动作训练、回放与评估\n",
"\n",
"目标：从公开三足步态入手，理解训练、checkpoint、回放与评估的区别。"
]
},
{
"cell_type": "markdown",
"id": "774872a9",
"metadata": {},
"source": [
"**实跑结果**：Ubuntu 工作站 GPU 从零训练 1000 次迭代，得到会前进的三足策略。最终模型在 0.3 m/s 指令下，两次 10 秒回放的平均前向速度为 0.2780 和 0.2759 m/s，均无重置。视频、代码、checkpoint 和评估数据见文末第 7 节；第 6 节保留一次没有走起来的续训对照。"
]
},
{
"cell_type": "markdown",
"id": "0f17da75",
"metadata": {},
"source": [
"## 使用方式\n",
"按顺序阅读并执行代码单元。默认单元只检查本机和演示原理，不会安装依赖、下载仓库或开始训练。下面的命令是供配置完成后手动执行的步骤；上游有更新时以对应版本文档为准。\n"
]
},
{
"cell_type": "markdown",
"id": "c6912c39",
"metadata": {},
"source": [
"## 1. 训练过程\n",
"\n",
"策略接收观测并输出动作；物理仿真产生下一状态，奖励和终止条件决定学习目标。三足步态任务重点关注速度指令跟踪。跳跃和舞蹈任务涉及参考动作，具体定义请阅读[上游教程](https://github.com/KingKongRobotics/jumper/blob/main/docs/TUTORIAL.md)。"
]
},
{
"cell_type": "markdown",
"id": "e7a55c46",
"metadata": {},
"source": [
"## 2. 配置完成后在 Jumper 根目录运行\n",
"\n",
"```bash\n",
"python scripts/train.py --list\n",
"python scripts/train.py --task jumper.tripod\n",
"python scripts/play.py --task jumper.tripod\n",
"```\n",
"\n",
"命令来自 [项目指南](https://github.com/KingKongRobotics/jumper/blob/main/docs/PROJECT_GUIDE.md)。先确认环境与显卡配置；训练可能耗时。回放默认选择的 checkpoint 和可用参数应以当前版本文档为准，记录具体文件与 commit。上游也提供 Colab 训练入口。"
]
},
{
"cell_type": "markdown",
"id": "b8089f33",
"metadata": {},
"source": [
"## 3. 设计可复核评估\n",
"\n",
"固定 checkpoint、任务、场景、种子集合、回合时长和成功判据；记录跌倒、速度跟踪误差及视频。多组实验比较时保持评估条件一致。以下是演示如何统计，不是实际实验结果。"
]
},
{
"cell_type": "code",
"execution_count": 1,
"id": "2f14666b",
"metadata": {
"execution": {
"iopub.execute_input": "2026-10-29T03:58:51.270477Z",
"iopub.status.busy": "2026-10-29T03:58:51.270477Z",
"iopub.status.idle": "2026-10-29T03:58:51.278008Z",
"shell.execute_reply": "2026-10-29T03:58:51.278008Z"
}
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"示例成功率: 3 / 5 = 0.6\n"
]
}
],
"source": [
"demo_success = [True, False, True, True, False]\n",
"print('示例成功率:', sum(demo_success), '/', len(demo_success), '=', sum(demo_success)/len(demo_success))"
]
},
{
"cell_type": "markdown",
"id": "0a6f48ec",
"metadata": {},
"source": [
"## 4. 从回放到部署\n",
"\n",
"阅读上游教程的动作包部分，理解策略、控制器及运行平台的对应关系；先在仿真验证，再按上游指南进行设备部署。\n",
"\n",
"**检查**：保存命令、环境、checkpoint、视频和评估结果到 [实验记录模板](../01-学习资料/实验记录模板.md)。短训成功启动只说明链路可运行，不能代替性能评估。"
]
},
{
"cell_type": "markdown",
"id": "49044e7b",
"metadata": {},
"source": [
"## 5. Ubuntu 工作站实跑：300 次 CPU 三足训练\n",
"\n",
"2026-10-29，Jumper 提交 `b2e58669848a3ba4f209597cd6bebdfe9d0533a4`；`native:cpu`、64 环境、4 物理线程，300 次迭代、460,800 环境步，日志所报训练时间 4 分 25 秒。最终 mean reward -5.29，线速度跟踪奖励 0.3666，课程等级 0；这些是训练日志指标，不是独立成功率。\n",
"\n",
"[完整日志](../01-学习资料/tripod-training.txt) · [详细复现记录](../01-学习资料/Ubuntu工作站复现记录.md) · [零动作基线视频](../assets/zero-baseline.mp4) · [训练后视频](../assets/tripod-299.mp4)"
]
},
{
"cell_type": "markdown",
"id": "a9f992cf",
"metadata": {},
"source": [
"### 回放与测量\n",
"\n",
"用最终 `model_299.pt`，固定机体前向指令 0.1 m/s，500 控制步、10 秒，观测噪声开启；单次回放没有 episode reset。平均前向速度约 -0.0013 m/s，平均绝对误差约 0.1013 m/s。该策略在此条件下基本没有跟踪前进指令，需要继续训练和评估。回放物理 1000 Hz、控制 50 Hz；仅是单次测量。"
]
},
{
"cell_type": "code",
"execution_count": 2,
"id": "49682047",
"metadata": {
"execution": {
"iopub.execute_input": "2026-10-29T03:58:51.281520Z",
"iopub.status.busy": "2026-10-29T03:58:51.281520Z",
"iopub.status.idle": "2026-10-29T03:58:51.534499Z",
"shell.execute_reply": "2026-10-29T03:58:51.534499Z"
}
},
"outputs": [
{
"data": {
"text/html": [
""
],
"text/plain": [
""
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"from IPython.display import Video, display\n",
"display(Video('../assets/tripod-299.mp4', embed=True, width=480))"
]
},
{
"cell_type": "code",
"execution_count": 3,
"id": "02097173",
"metadata": {
"execution": {
"iopub.execute_input": "2026-10-29T03:58:51.543670Z",
"iopub.status.busy": "2026-10-29T03:58:51.543164Z",
"iopub.status.idle": "2026-10-29T03:58:51.549713Z",
"shell.execute_reply": "2026-10-29T03:58:51.549713Z"
}
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"command: [0.1, 0.0, 0.0]\n",
"control_steps: 500\n",
"simulated_seconds: 10.0\n",
"episode_resets: 0\n",
"actual_velocity_mean: [-0.0013081777529005195, -0.0004899571883943281, -0.0036823606009420474]\n",
"mean_absolute_error: [0.10130817775290142, 0.006883609439239081, 0.059962322468782074]\n",
"observation_noise: True\n"
]
}
],
"source": [
"import json\n",
"from pathlib import Path\n",
"evaluation = json.loads(Path('../01-学习资料/evaluation.json').read_text(encoding='utf-8'))\n",
"for key in ('command', 'control_steps', 'simulated_seconds', 'episode_resets', 'actual_velocity_mean', 'mean_absolute_error', 'observation_noise'):\n",
" print(f'{key}: {evaluation[key]}')"
]
},
{
"cell_type": "markdown",
"id": "91eb9a15",
"metadata": {},
"source": [
"\n",
"\n",
"[测量 CSV](../01-学习资料/measure.csv) · [评估 JSON](../01-学习资料/evaluation.json) · [回放配置](../01-学习资料/replay-info.json)\n",
"\n",
"训练和回放的精确命令、checkpoint 路径与哈希见复现记录。视频呈现一次仿真轨迹，不能替代多种子评估。"
]
},
{
"cell_type": "markdown",
"id": "b29a17d5",
"metadata": {},
"source": [
"## 6. GPU 续训与实际排查\n",
"\n",
"从 CPU `model_299.pt` 恢复，使用 Warp CUDA、4096 个环境再训练 1000 次迭代；日志记录 98,304,000 个环境步、11 分 29 秒。最后 mean reward 28.79，课程等级 1。\n",
"\n",
"对最终 `model_1298.pt` 以 0.3 m/s、200 Hz 物理、50 Hz 控制、观测噪声开启回放 10 秒：没有 reset，但平均前向速度 -0.00080 m/s，前向平均绝对误差 0.30080 m/s，策略仍未有效前进。"
]
},
{
"cell_type": "code",
"execution_count": 4,
"id": "fcdd5e5e",
"metadata": {
"execution": {
"iopub.execute_input": "2026-10-29T03:58:51.552720Z",
"iopub.status.busy": "2026-10-29T03:58:51.552720Z",
"iopub.status.idle": "2026-10-29T03:58:51.564005Z",
"shell.execute_reply": "2026-10-29T03:58:51.564005Z"
}
},
"outputs": [
{
"data": {
"text/html": [
""
],
"text/plain": [
""
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"display(Video('../assets/tripod-gpu-final.mp4', embed=True, width=480))"
]
},
{
"cell_type": "code",
"execution_count": 5,
"id": "de425367",
"metadata": {
"execution": {
"iopub.execute_input": "2026-10-29T03:58:51.575139Z",
"iopub.status.busy": "2026-10-29T03:58:51.573634Z",
"iopub.status.idle": "2026-10-29T03:58:51.581429Z",
"shell.execute_reply": "2026-10-29T03:58:51.581429Z"
}
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"command: [0.3, 0.0, 0.0]\n",
"physics_hz: 200.0\n",
"control_steps: 500\n",
"episode_resets: 0\n",
"actual_velocity_mean: [-0.0007965899825212546, 0.0018699594324571081, 0.01917030018335208]\n",
"mean_absolute_error: [0.30079658998252123, 0.007546386151516344, 0.048324697711970656]\n",
"observation_noise: True\n"
]
}
],
"source": [
"gpu_result = json.loads(Path('../01-学习资料/gpu-final-evaluation.json').read_text(encoding='utf-8'))\n",
"for key in ('command', 'physics_hz', 'control_steps', 'episode_resets', 'actual_velocity_mean', 'mean_absolute_error', 'observation_noise'):\n",
" print(f'{key}: {gpu_result[key]}')"
]
},
{
"cell_type": "markdown",
"id": "aa08d0dc",
"metadata": {},
"source": [
"中间 checkpoint 在 200/1000 Hz、CPU/GPU 后端下均接近原地；命令管理器检查显示它收到 0.3 m/s，但不能据此认定整个观测链都没有问题。[复现记录与诊断](../01-学习资料/Ubuntu工作站复现记录.md)包含命令、数据和 checkpoint 哈希。[官方预训练动作包演示](05_官方动作包_跳跃与舞蹈.ipynb)展示实际行走、跳跃和舞蹈，来源与本次训练不同。"
]
},
{
"cell_type": "markdown",
"id": "57e38029",
"metadata": {},
"source": [
"## 7. GPU 从零训练：自训三足策略行走\n",
"\n",
"保持上游任务配置，从随机初始化以 Warp CUDA、4096 环境训练。`model_200.pt` 在 CPU 原生后端、200 Hz 物理、50 Hz 控制、观测噪声开启的 10 秒回放中，收到 0.3 m/s 前向指令，平均前向速度 **0.2724 m/s**，没有 reset。录视频的另一次独立回放为 0.2878 m/s；1000 Hz 物理回放为 0.2793 m/s。回放未固定种子，结果随运行变化。"
]
},
{
"cell_type": "code",
"execution_count": 6,
"id": "2491b60d",
"metadata": {
"execution": {
"iopub.execute_input": "2026-10-29T03:58:51.585278Z",
"iopub.status.busy": "2026-10-29T03:58:51.585278Z",
"iopub.status.idle": "2026-10-29T03:58:51.600676Z",
"shell.execute_reply": "2026-10-29T03:58:51.600676Z"
}
},
"outputs": [
{
"data": {
"text/html": [
""
],
"text/plain": [
""
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"display(Video('../assets/tripod-fresh200.mp4', embed=True, width=480))"
]
},
{
"cell_type": "code",
"execution_count": 7,
"id": "028be57c",
"metadata": {
"execution": {
"iopub.execute_input": "2026-10-29T03:58:51.619703Z",
"iopub.status.busy": "2026-10-29T03:58:51.618698Z",
"iopub.status.idle": "2026-10-29T03:58:51.626600Z",
"shell.execute_reply": "2026-10-29T03:58:51.626600Z"
}
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"fresh200-eval.json physics Hz: 200.0 vx: 0.2724 resets: 0\n",
"fresh200-video-evaluation.json physics Hz: 200.0 vx: 0.2878 resets: 0\n",
"fresh200-cpu1000-03.json physics Hz: 1000.0 vx: 0.2793 resets: 0\n"
]
}
],
"source": [
"for name in ('fresh200-eval.json', 'fresh200-video-evaluation.json', 'fresh200-cpu1000-03.json'):\n",
" result = json.loads((Path('../01-学习资料') / name).read_text(encoding='utf-8'))\n",
" print(name, 'physics Hz:', result['physics_hz'], 'vx:', round(result['actual_velocity_mean'][0], 4), 'resets:', result['episode_resets'])"
]
},
{
"cell_type": "markdown",
"id": "06176d82",
"metadata": {},
"source": [
"### 1000 次迭代后的最终模型\n",
"\n",
"完成 1000 次迭代，共 98,304,000 环境步。下方视频和数据来自最终 `model_999.pt`，使用 CPU 原生后端、200 Hz 物理、观测噪声开启；每次回放 10 秒。"
]
},
{
"cell_type": "code",
"execution_count": 8,
"id": "4daf0f0c",
"metadata": {
"execution": {
"iopub.execute_input": "2026-10-29T03:58:51.630608Z",
"iopub.status.busy": "2026-10-29T03:58:51.630608Z",
"iopub.status.idle": "2026-10-29T03:58:51.646027Z",
"shell.execute_reply": "2026-10-29T03:58:51.646027Z"
}
},
"outputs": [
{
"data": {
"text/html": [
""
],
"text/plain": [
""
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"display(Video('../assets/tripod-fresh999.mp4', embed=True, width=480))"
]
},
{
"cell_type": "code",
"execution_count": 9,
"id": "c062ccc7",
"metadata": {
"execution": {
"iopub.execute_input": "2026-10-29T03:58:51.661541Z",
"iopub.status.busy": "2026-10-29T03:58:51.661541Z",
"iopub.status.idle": "2026-10-29T03:58:51.669877Z",
"shell.execute_reply": "2026-10-29T03:58:51.669877Z"
}
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"fresh999-repeat.json target: 0.3 vx: 0.278 MAE: 0.022 resets: 0\n",
"fresh999-video.json target: 0.3 vx: 0.2759 MAE: 0.0243 resets: 0\n",
"fresh999-fast.json target: 0.5 vx: 0.4593 MAE: 0.0407 resets: 0\n"
]
}
],
"source": [
"for name in ('fresh999-repeat.json', 'fresh999-video.json', 'fresh999-fast.json'):\n",
" result = json.loads((Path('../01-学习资料') / name).read_text(encoding='utf-8'))\n",
" print(name, 'target:', result['command'][0], 'vx:', round(result['actual_velocity_mean'][0], 4), 'MAE:', round(result['mean_absolute_error'][0], 4), 'resets:', result['episode_resets'])"
]
},
{
"cell_type": "markdown",
"id": "1de1de7a",
"metadata": {},
"source": [
"\n",
"\n",
"[训练与回放命令、checkpoint 哈希](../01-学习资料/Ubuntu工作站复现记录.md) · [最终自训 checkpoint](../02-可运行代码/checkpoints/tripod-gpu-fresh999.pt) · [测量 CSV](../01-学习资料/fresh999-measure.csv)。多次回放没有固定种子，结果说明策略在这些指定仿真条件下能行走；真机效果尚未测量。"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.12.3"
}
},
"nbformat": 4,
"nbformat_minor": 5
}