Star 历史趋势
数据来源: GitHub API · 生成自 Stargazers.cn
README.md

vLLM Recipes

This repo intends to host community maintained common recipes to run vLLM answering the question: How do I run model X on hardware Y for task Z?

Guides

Arcee AI Arcee AI

DeepSeek DeepSeek

Ernie Ernie

GLM GLM

Google Google

inclusionAI inclusionAI

InternVL InternVL

InternLM InternLM

Jina AI Jina AI

Llama

Microsoft Microsoft

MiniMax minmax

Xiaomi MiMo Xiaomi MiMo

Mistral AI Mistral AI

Moonshotai Moonshotai

NVIDIA

OpenAI OpenAI

PaddlePaddle PaddlePaddle

Qwen Qwen

Seed Seed

StepFun StepFun

Tencent-Hunyuan Tencent-Hunyuan

How recipes are maintained

Each recipe is a structured YAML file at models/<hf_org>/<hf_repo>.yaml, kept in the repository of the project that knows the model best. This repository is the renderer: it pulls those files in, merges them, and builds recipes.vllm.ai and its JSON API.

flowchart LR
  vllm["vllm-project/vllm<br/>recipes/models/"] --> sync
  metal["vllm-metal<br/>recipes/overlays/"] --> sync
  omni["vllm-project/vllm-omni<br/>recipes/models/"] --> sync
  ascend["vllm-ascend<br/>overlays/"] -.-> sync
  self["this repo<br/>taxonomy · strategies · kv_store"] --> assemble
  sync["sync-sources.mjs<br/>pinned by sources.lock.json"] --> assemble["assemble-content.mjs<br/>→ .content/&lt;channel&gt;/"]
  assemble --> build["build-recipes-api.mjs + next build"] --> site["recipes.vllm.ai<br/>+ JSON API"]
ContentWhere it livesStatus
Model recipes served by vLLMvllm-project/vllm → recipes/models/Owned by vLLM; a merge there reaches recipes.vllm.ai in about 30 minutes. Copies stay in this repo's models/ until they are synced to vLLM and removed; they aren't rendered
Omni recipes (image, video, audio generation)vllm-project/vllm-omni → recipes/models/Owned by vLLM-Omni, reaching recipes.vllm.ai the same way. Copies stay in this repo's models/ until they are removed; they aren't rendered
Apple Silicon variantsvllm-project/vllm-metal → recipes/overlays/Metal adds variants to the original model pages; featured.yaml controls its own docs only
Ascend NPU contentvllm-project/vllm-ascend → recipes/overlays/Owned by vLLM Ascend; merged into the vLLM recipes, whose source rejects Huawei hardware
Schema, hardware profiles, serving strategies, KV-store deployments, platformsThis repo (taxonomy.yaml, strategies/, kv_store/, platforms.yaml)Owned here
Command synthesis, site, JSON APIThis repo (src/, scripts/)Owned here

How the pieces fit:

  • sources.yaml lists every source repository for a channel (today only latest). For each source it gives the repo, ref, and directory, a role, and the scope this repo grants it:

    • core / owner sources ship whole recipes. Each Hugging Face ID may come from only one source.
    • overlay sources add content to another source's recipe, for example Ascend profiles, variants, and overrides from vllm-ascend. An overlay can only add keys, and only for the hardware brands its scope grants, so it can never change what the NVIDIA or AMD commands render.
  • sources.lock.json pins every remote source to a commit, so each build is reproducible.

  • Updates are automatic. bump-sources.yml runs every 20 minutes:

    1. Checks whether a source's recipe directory changed.
    2. Validates the new content with the full JSON API build.
    3. Commits the lock, which redeploys the site.

    If validation fails, the lock stays on the last good commit, so the site keeps serving the previous content, and a sources-bump-failed issue is opened. There is one issue per failure, not one per run, and it closes itself on the next successful run.

  • Provenance. Every recipe page links "Edit recipe" to the repository that owns the file. Every recipe JSON carries a source field (repo, commit, path) for tracing a command back to its YAML.

Contributing

Edit or add a recipe in the repository that owns it (see the table above), following CONTRIBUTING.md for the full schema, VRAM formula, and validation steps. Changes to the schema, hardware profiles, strategies, or the site go to this repository.

To preview a recipe change from a local checkout of a source repository, create a gitignored sources.local.yaml in this repo:

overrides:
  vllm: ~/vllm/recipes
  vllm-omni: ~/vllm-omni/recipes

Quick loop:

pnpm install
pnpm dev                              # http://localhost:3000
node scripts/sync-sources.mjs         # fetch the source commits pinned in sources.lock.json
node scripts/build-recipes-api.mjs    # assembles sources, validates every YAML, rebuilds the JSON API
node scripts/validate-source.mjs --source vllm --dir ~/vllm/recipes   # validate a source checkout (what its CI runs)
pnpm content:watch                    # re-assemble when files are added to or removed from a local source

Legacy MkDocs guides

The top-level Markdown directories (DeepSeek/, Qwen/, etc.) are the historical MkDocs site, kept as a reference during the YAML migration. To preview them:

uv venv
source .venv/bin/activate
uv pip install -r requirements.txt
uv run mkdocs serve --dev-addr 127.0.0.1:8001

License

This project is licensed under the Apache License 2.0 - see the LICENSE file for details.

关于 About

Common recipes to run vLLM

语言 Languages

JavaScript93.4%
Jupyter Notebook4.5%
CSS0.9%
Python0.9%
HTML0.3%
Shell0.1%

提交活跃度 Commit Activity

代码提交热力图
过去 52 周的开发活跃度
939
Total Commits
峰值: 51次/周
Less
More

核心贡献者 Contributors