Relational Transformer (RT)

This repo is the official implementation of the Relational Transformer (RT) architecture and the recipes to pretrain, fine-tune, run inference with and test-time scale Relational Foundation Models (RFMs) based on RT.

The current state of the repo reflects our NeurIPS 2026 work, RT-J, whose pretraining recipe is shown below:

This repo is linked to the following papers in the Stanford Tabular and Relational (STAR) project ecosystem:

Models and datasets (including preprocessed versions) are available on Hugging Face. See also our repo on agentic task curation for the Join corpus.

Installation

pip install relational-transformer

This installs the rt package, including the Rust sampler rt.rustler, without any dependency on Rust.

Quickstart

The Quickstart notebook (open in Colab) evaluates the released RT-J checkpoint on a RelBench task, rel-f1/driver-dnf, with no training: download one preprocessed database, predict every test row from its sampled context, score the AUROC (0.826 on an A100). A few minutes on an A100 or L4 runtime.

Bring your own data (BYOD)

RT runs on any database in RelBench format: Parquet tables, a manifest.yaml naming their keys and time columns, and labelled splits per task. See RelBench's BYOD guide for the format. The BYOD notebook (open in Colab) builds one from a DuckDB database with RelBench, with tasks defined in SQL, then preprocesses it and predicts with a released checkpoint. For more than a first look, examples/ runs the full recipes on it: in-context prediction with a frozen checkpoint, or per-task fine-tuning.

Preprocessing a large database (text embeddings included) can take a while, so smoke test on a subsample of it first to get to a first prediction quickly.

Development

We use pixi to manage one self-contained environment (Python, PyTorch + CUDA, Rust, and all dependencies), built on first use. The environment is defined for Linux x86_64 only.

git clone https://github.com/stanford-star/relational-transformer.git
cd relational-transformer
pixi run python -m examples.eval.plan        # or examples.pretrain.phases, ...

Examples

examples/ holds the recipes behind the released checkpoints and leaderboard entries, with the notes on how each part works.

Citation

If you use this repo, please cite the following papers:

@inproceedings{ranjan2026rtj,
    title={{RT-J}: Large-Scale Pretraining of Relational Transformers for Context-Efficient Predictions},
    author={Rishabh Ranjan and Vignesh Kothapalli and Harshvardhan Agarwal and Charilaos Kanatsoulis and Roshan Upendra and Tom Palczewski and Carlos Guestrin and Jure Leskovec},
    booktitle={The Fortieth Annual Conference on Neural Information Processing Systems},
    year={2026}
}

@inproceedings{ranjan2026relational,
    title={Relational Transformer: Toward Zero-Shot Foundation Models for Relational Data},
    author={Rishabh Ranjan and Valter Hudovernik and Mark Znidar and Charilaos Kanatsoulis and Roshan Upendra and Mahmoud Mohammadi and Joe Meyer and Tom Palczewski and Carlos Guestrin and Jure Leskovec},
    booktitle={The Fourteenth International Conference on Learning Representations},
    year={2026}
}

Additionally, if you use RT-PluRel please also cite:

@inproceedings{kothapalli2026plurel,
    title={{PluRel}: Synthetic Data unlocks Scaling Laws for Relational Foundation Models},
    author={Vignesh Kothapalli and Rishabh Ranjan and Valter Hudovernik and Vijay Prakash Dwivedi and Johannes Hoffart and Carlos Guestrin and Jure Leskovec},
    booktitle={Forty-third International Conference on Machine Learning},
    year={2026}
}

License

Apache-2.0, see LICENSE. The released checkpoints and datasets carry their own licences; "the Join" is assembled from third-party databases that keep theirs.