cff-version: 1.2.0
title: "miniVERL: Run verl experiment semantics on one GPU"
message: "If you use miniVERL in your work, please cite it as below."
type: software
version: 0.14.0
date-released: 2026-09-09
license: Apache-2.0
repository-code: "https://github.com/DaoyuanLi2816/mini-verl"
url: "https://github.com/DaoyuanLi2816/mini-verl"
abstract: >-
  miniVERL compiles supported verl experiment semantics for one NVIDIA GPU.
  Its versioned profiles consume resolved configuration and Parquet prompts,
  run PPO with an independent critic or critic-free reward-driven algorithms,
  and support on-policy distillation with explicit teachers. Typed trajectories,
  role identities, transactional checkpoints and exact replay make local runs
  inspectable and resumable. Portable PEFT, critic, Parquet and provenance
  bundles support handoff toward pinned upstream verl. Installed PPO and GRPO
  walkthroughs exercise the complete workflow. Runtime qualification is measured
  on one RTX 4080 and kept separate from task-quality research; distributed
  execution is outside the local runtime. Frozen studies preserve their original
  negative and mixed results.
authors:
  - family-names: Li
    given-names: Daoyuan
    email: lidaoyuan2816@gmail.com
keywords:
  - on-policy distillation
  - knowledge distillation
  - llm alignment
  - preference optimization
  - large language models
  - tool use
  - agents
  - PEFT
  - QLoRA
  - consumer GPU
  - single GPU
  - post-training
  - reproducibility
references:
  - type: article
    title: "Distilling the Knowledge in a Neural Network"
    authors:
      - family-names: Hinton
        given-names: Geoffrey
      - family-names: Vinyals
        given-names: Oriol
      - family-names: Dean
        given-names: Jeff
    year: 2015
    url: "https://arxiv.org/abs/1503.02531"
    notes: "Source of the high-temperature forward-KL correction; reverse-KL and JSD use it only as an explicit heuristic."
  - type: article
    title: >-
      On-Policy Distillation of Language Models: Learning from Self-Generated
      Mistakes
    authors:
      - family-names: Agarwal
        given-names: Rishabh
    year: 2024
    url: "https://arxiv.org/abs/2306.13649"
    notes: "Generalized JSD and training on student-sampled sequences."
  - type: article
    title: "On-Policy Context Distillation for Language Models"
    authors:
      - family-names: Ye
        given-names: Tianzhu
      - family-names: Dong
        given-names: Li
      - family-names: Wu
        given-names: Xun
      - family-names: Huang
        given-names: Shaohan
      - family-names: Wei
        given-names: Furu
    year: 2026
    url: "https://arxiv.org/abs/2602.12275"
    notes: "The privileged-context teacher mode follows this formulation."
  - type: article
    title: "Entropy-Aware On-Policy Distillation of Language Models"
    authors:
      - family-names: Jin
        given-names: Woogyeol
      - family-names: Min
        given-names: Taywon
      - family-names: Yang
        given-names: Yongjin
      - family-names: Wei
        given-names: Dennis
      - family-names: Zhou
        given-names: Yi
      - family-names: Kadhe
        given-names: Swanand Ravindra
      - family-names: Baracaldo
        given-names: Nathalie
      - family-names: Lee
        given-names: Kimin
    year: 2026
    url: "https://arxiv.org/abs/2603.07079"
    notes: >-
      Motivates recording per-token teacher entropy. Entropy-aware divergence
      mixing is a roadmap item and is not implemented in miniVERL.