τ₀-VLA: a Hierarchical Robot Foundation Model with World-Model-Guided Test-Time Computation

This repo is the official implementation of τ₀-VLA: a Hierarchical Robot Foundation Model with World-Model-Guided Test-Time Computation.

News

• [2026.07.27] 🚀 We release the τ₀-VLA model Paper, Project Website, and Hugging Face.

Overview

τ₀-VLA is a hierarchical robot foundation model for long-horizon manipulation. A memory-augmented high-level policy generates the next subtask and uses world-model-guided test-time computation to search over alternatives when additional reasoning is needed. A generalist low-level policy then executes the selected subtask across robot embodiments.

The low-level policy combines a Qwen3.5 vision-language backbone with a Mixture-of-Transformers action expert trained through conditional flow matching. It uses a unified 40-dimensional state/action space and was trained on 40,115 hours of heterogeneous real-world robot data with multimodal co-training.

Installation

The reference environment uses Python 3.11, CUDA 12.8, and PyTorch 2.7.1.

git clone git@github.com:sii-research/tau-0-vla.git
cd tau-0-vla
bash scripts/setup.sh

Example data and post-training

example_data/ contains a small AgiBot World subset in LeRobot v3.0 format. The matching post-training recipe is under configs/example_agibot_world_gong/.

bash scripts/train.sh configs/example_agibot_world_gong/train.yaml \
    --model_name_or_path /path/to/tau-0-vla-checkpoint

For another dataset or robot, start from configs/_template/ and src/tau0_vla/adapters/_template/.

Serving and evaluation

Public v1 serving supports joint-control checkpoints only. Native EEF data may be used for training, but EEF serving is not supported in this release.

Serve a post-trained joint-control checkpoint:

python -m deploy.server --model outputs/<run_name>

Run open-loop evaluation:

python deploy/openloop.py --ckpt outputs/<run_name> --no-plot

See deploy/ for the payload and action-order contracts.

Repository layout

src/tau0_vla/
├── adapters/    embodiment-specific data layouts and deployment I/O
├── data/        LeRobot loading, prompting, masking, and normalization
├── models/      Qwen3.5 backbone and flow-matching action expert
├── trainer/     post-training entry point
├── vlm/         multimodal collation and tokenization
└── utils/       logging and run specifications
configs/         reusable template and the AgiBot World example
deploy/          policy server and open-loop evaluation
example_data/    bundled AgiBot World subset
scripts/         setup, training, and normalization utilities

Additional documentation:

• Dataset format
• Data pipeline
• Robot adapters

License

Code and model weights are released under the Apache License 2.0. The bundled example data follows the license described in example_data/README.md.