{ "cells": [ { "cell_type": "markdown", "id": "c5ec08d9", "metadata": {}, "source": [ "# 07｜舞蹈：动作参考、策略训练与仿真回放\n", "\n", "使用上游开放的舞蹈轨迹和训练代码，学习动作跟踪与训练评估；下方视频为本专题本次运行保存的结果。" ] }, { "cell_type": "markdown", "id": "199a6cf3", "metadata": {}, "source": [ "## 运行前提\n", "\n", "本文记录的上游代码来自 [KingKongRobotics/jumper](https://github.com/KingKongRobotics/jumper)，提交 `b2e58669848a3ba4f209597cd6bebdfe9d0533a4`。完整环境检查和旧实验见 [Ubuntu 工作站复现记录](../01-学习资料/Ubuntu工作站复现记录.md)。下方代码单元读取已经保存的结果并显示视频，不会自动安装依赖或启动训练。\n", "\n", "工作站代码在 `/home/ubuntu/projects/robotics_shared/repos/jumper-everyembodied`，Python 为 `/data/Data14TB/envs/jumper-everyembodied/bin/python`。下方命令在仓库根目录执行；`warp:cuda` 用于 GPU 训练，`native:cpu` 和 OSMesa 用于无窗口录像。训练日志、checkpoint、视频和评估数据来自不同步骤，应在报告中分别记录。具体版本和结果以本专题保存的证据为准。\n" ] }, { "cell_type": "markdown", "id": "c06a0b58", "metadata": {}, "source": [ "## 1. 已公开的内容\n", "\n", "[舞蹈任务](https://github.com/KingKongRobotics/jumper/tree/b2e58669848a3ba4f209597cd6bebdfe9d0533a4/tasks/jumper/dance) 包含参考动作 `media/demo.npz`、任务配置与 PPO 训练代码。首次构建时会上游转换和缓存参考数据。仓库中的 `demo.mp4` 是脸部动画素材，不是本次自训机器人视频。训练不需要音乐；输出配乐成片依赖上游未提供的音乐，这里仅使用无声仿真回放。" ] }, { "cell_type": "markdown", "id": "2dc121ac", "metadata": {}, "source": [ "## 2. PPO 跟踪参考动作\n", "\n", "策略基于机器人状态和参考动作信息输出关节控制；奖励衡量对参考运动的跟踪及运动质量。上游使用动作跟踪 runner，记录位置、姿态等指标；对舞蹈而言，能否持续完成参考片段和是否出现终止也应结合视频判断。模型、参考文件和代码版本应一同记录。" ] }, { "cell_type": "markdown", "id": "ecfbd36f", "metadata": {}, "source": [ "## 3. 在工作站训练\n", "\n", "```bash\n", "cd /home/ubuntu/projects/robotics_shared/repos/jumper-everyembodied\n", "PY=/data/Data14TB/envs/jumper-everyembodied/bin/python\n", "\"$PY\" scripts/train.py --task jumper.dance --dry-run\n", "OMP_NUM_THREADS=2 \"$PY\" scripts/train.py --task jumper.dance --backend warp --device cuda:0 --num_envs 1024 --max-iterations 500 --headless --no-tensorboard\n", "```\n", "\n", "该命令是可调整迭代数的工作站入口；实际使用的命令和结果应以本专题的训练日志及 JSON 为准。上游配置默认 10000 次迭代。\n", "\n", "本次与其他 GPU 任务共享工作站，采用 1024 环境，改变了 PPO 批量；上游 GPU 示例为 4096 环境。" ] }, { "cell_type": "markdown", "id": "00e5790b", "metadata": {}, "source": [ "## 4. 检查与录像\n", "\n", "观察训练日志里的参考动作跟踪、位置和方向误差以及终止信息；高训练奖励不等于完整表演。录像前确认 checkpoint、参考素材和任务一致，用有限步数与 `--stop-on-done` 保留失败终止。\n", "\n", "```bash\n", "CKPT=logs/jumper/jumper.dance/2026-10-29_14-54-46/model_200.pt\n", "MUJOCO_GL=osmesa OMP_NUM_THREADS=1 \"$PY\" scripts/play.py --task jumper.dance --backend native --device cpu --checkpoint \"$CKPT\" --headless --steps 1000 --physics-hz 200 --stop-on-done --measure --video dance.mp4 --video-width 640 --video-height 480 --speed 0\n", "```\n", "\n", "确认视频实际时长和是否提前终止。无音乐不妨碍检查关节动作；播放过的官方动作包应明确与自训模型区分。\n", "\n", "上游参考舞蹈约 235.7 秒；本片 20 秒、无配乐，属于开头片段。第 50 和 100 次 checkpoint 的回放在第 2 和 3 步终止，第 200 次的一次回放连续运行 1000 步。训练日志中的关节位置误差是关节向量 L2 范数（rad），不是逐关节平均或 RMS。" ] }, { "cell_type": "markdown", "id": "d8e3b1fd", "metadata": {}, "source": [ "## 5. 本专题实测数据" ] }, { "cell_type": "code", "execution_count": 1, "id": "1c0db880", "metadata": { "execution": { "iopub.execute_input": "2026-10-29T07:13:32.119361Z", "iopub.status.busy": "2026-10-29T07:13:32.119361Z", "iopub.status.idle": "2026-10-29T07:13:32.127147Z", "shell.execute_reply": "2026-10-29T07:13:32.127147Z" } }, "outputs": [], "source": [ "import json\n", "from pathlib import Path\n", "from IPython.display import Video, display\n", "\n", "evidence_dir = Path('../01-学习资料')\n", "media_dir = Path('../assets')\n" ] }, { "cell_type": "code", "execution_count": 2, "id": "72ddd847", "metadata": { "execution": { "iopub.execute_input": "2026-10-29T07:13:32.131156Z", "iopub.status.busy": "2026-10-29T07:13:32.131156Z", "iopub.status.idle": "2026-10-29T07:13:32.138537Z", "shell.execute_reply": "2026-10-29T07:13:32.138537Z" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "{\n", " \"task\": \"jumper.dance\",\n", " \"upstream_commit\": \"b2e58669848a3ba4f209597cd6bebdfe9d0533a4\",\n", " \"training\": {\n", " \"backend\": \"warp:cuda\",\n", " \"num_envs\": 1024,\n", " \"checkpoint_iteration\": 200,\n", " \"seed\": null,\n", " \"reference_duration_s_approx\": 235.7\n", " },\n", " \"checkpoint\": \"../02-可运行代码/checkpoints/dance-model-200.pt\",\n", " \"checkpoint_sha256\": \"1f68c3014282633f54d5628fcb55786eb41d8a292dfe970fb6eec99324614b11\",\n", " \"training_metrics_at_iteration_200\": {\n", " \"mean_reward\": 0.92,\n", " \"joint_position_error_l2_rad\": 1.7688,\n", " \"mean_body_position_error_m\": 0.0642\n", " },\n", " \"evaluation\": {\n", " \"backend\": \"native:cpu\",\n", " \"physics_hz\": 200,\n", " \"control_hz\": 50,\n", " \"observation_noise\": true,\n", " \"steps_requested\": 1000,\n", " \"steps_completed\": 1000,\n", " \"early_termination\": false,\n", " \"duration_s\": 20.0,\n", " \"note\": \"A short reference-start replay, not the full choreography.\"\n", " },\n", " \"video\": \"../assets/dance-trained.mp4\"\n", "}\n" ] } ], "source": [ "result_path = evidence_dir / 'dance-result.json'\n", "if result_path.is_file():\n", " result = json.loads(result_path.read_text(encoding=\"utf-8\"))\n", " print(json.dumps(result, ensure_ascii=False, indent=2))\n", "else:\n", " print(\"尚未找到实际评估数据:\", result_path)\n" ] }, { "cell_type": "markdown", "id": "ae82c8c2", "metadata": {}, "source": [ "## 6. 自训策略视频\n", "\n", "视频和评估文件需标注同一 checkpoint、轨迹与回放条件。" ] }, { "cell_type": "code", "execution_count": 3, "id": "c9c4bdad", "metadata": { "execution": { "iopub.execute_input": "2026-10-29T07:13:32.142546Z", "iopub.status.busy": "2026-10-29T07:13:32.142546Z", "iopub.status.idle": "2026-10-29T07:13:32.416583Z", "shell.execute_reply": "2026-10-29T07:13:32.416077Z" } }, "outputs": [ { "data": { "text/html": [ "" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "video_path = media_dir / 'dance-trained.mp4'\n", "if video_path.is_file():\n", " display(Video(str(video_path), embed=True, width=640))\n", "else:\n", " print(\"尚未找到实际视频:\", video_path)\n" ] }, { "cell_type": "markdown", "id": "5ab70a4f", "metadata": {}, "source": [ "## 延伸阅读\n", "\n", "- [舞蹈参考数据说明](https://github.com/KingKongRobotics/jumper/blob/b2e58669848a3ba4f209597cd6bebdfe9d0533a4/tasks/jumper/dance/media/README.md)\n", "- [上游动作包训练教程](https://github.com/KingKongRobotics/jumper/blob/b2e58669848a3ba4f209597cd6bebdfe9d0533a4/docs/TUTORIAL.md)\n", "- [专题首页](../README.md) · [跳跃任务](06_跳跃_参考动作训练与评估.ipynb) · [官方动作包演示](05_官方动作包_跳跃与舞蹈.ipynb)\n", "\n", "第三方素材许可见上游 `NOTICE`；仿真结果与真机执行分别记录。" ] }, { "cell_type": "markdown", "id": "1ee7029c", "metadata": {}, "source": [ "## 7. 500 次训练后的最终模型\n", "\n", "使用最终 `model_499.pt` 重新回放，同样在 200 Hz 物理、50 Hz 控制及观测噪声开启时完成 1000 步、20 秒，没有提前终止。训练日志中的第 499 次关节位置误差 L2 为 0.3505 rad，刚体平均位置误差为 0.0123 m；这些是训练批量指标，不是录像回放误差。" ] }, { "cell_type": "code", "execution_count": 4, "id": "bb20c80e", "metadata": { "execution": { "iopub.execute_input": "2026-10-29T07:13:32.436167Z", "iopub.status.busy": "2026-10-29T07:13:32.436167Z", "iopub.status.idle": "2026-10-29T07:13:32.443186Z", "shell.execute_reply": "2026-10-29T07:13:32.443186Z" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "{\n", " \"task\": \"jumper.dance\",\n", " \"upstream_commit\": \"b2e58669848a3ba4f209597cd6bebdfe9d0533a4\",\n", " \"training\": {\n", " \"backend\": \"warp:cuda\",\n", " \"num_envs\": 1024,\n", " \"iterations_completed\": 500,\n", " \"environment_steps\": 12288000,\n", " \"training_time\": \"00:10:34\",\n", " \"seed\": null,\n", " \"reference_duration_s_approx\": 235.7\n", " },\n", " \"checkpoint\": \"../02-可运行代码/checkpoints/dance-model-499.pt\",\n", " \"checkpoint_sha256\": \"5791a7797c4519e60b3e5b92a780c80611559b66e4f8b59c72502f984a83f0a7\",\n", " \"training_metrics_at_iteration_499\": {\n", " \"mean_reward\": 41.53,\n", " \"joint_position_error_l2_rad\": 0.3505,\n", " \"mean_body_position_error_m\": 0.0123\n", " },\n", " \"evaluation\": {\n", " \"backend\": \"native:cpu\",\n", " \"physics_hz\": 200,\n", " \"control_hz\": 50,\n", " \"observation_noise\": true,\n", " \"steps_requested\": 1000,\n", " \"steps_completed\": 1000,\n", " \"early_termination\": false,\n", " \"duration_s\": 20.0,\n", " \"note\": \"A short reference-start replay, not the full choreography.\"\n", " },\n", " \"video\": \"../assets/dance-final.mp4\"\n", "}\n" ] } ], "source": [ "result_path = evidence_dir / 'dance-final-result.json'\n", "if result_path.is_file():\n", " result = json.loads(result_path.read_text(encoding=\"utf-8\"))\n", " print(json.dumps(result, ensure_ascii=False, indent=2))\n", "else:\n", " print(\"尚未找到实际评估数据:\", result_path)\n" ] }, { "cell_type": "code", "execution_count": 5, "id": "ebe34e67", "metadata": { "execution": { "iopub.execute_input": "2026-10-29T07:13:32.446711Z", "iopub.status.busy": "2026-10-29T07:13:32.446711Z", "iopub.status.idle": "2026-10-29T07:13:32.461895Z", "shell.execute_reply": "2026-10-29T07:13:32.461895Z" } }, "outputs": [ { "data": { "text/html": [ "" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "video_path = media_dir / 'dance-final.mp4'\n", "if video_path.is_file():\n", " display(Video(str(video_path), embed=True, width=640))\n", "else:\n", " print(\"尚未找到实际视频:\", video_path)\n" ] }, { "cell_type": "markdown", "id": "ebd313e8", "metadata": {}, "source": [ "[完整训练与评估记录](../01-学习资料/跳跃与舞蹈实跑记录.md) · [训练日志](../01-学习资料/dance-training.txt) · [最终回放日志](../01-学习资料/dance-final-replay.txt) · [最终 checkpoint](../02-可运行代码/checkpoints/dance-model-499.pt)\n", "\n", "最终模型重放命令与第 200 次一致，将 `CKPT` 改为 `logs/jumper/jumper.dance/2026-10-29_14-54-46/model_499.pt`。本次使用上游开源参考与算法；没有公开的配乐不包含在成果中。" ] } ], "metadata": { "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.12.3" } }, "nbformat": 4, "nbformat_minor": 5 }