Star 历史趋势
数据来源: GitHub API · 生成自 Stargazers.cn
README.md

Open Weight Models

A curated list of open-weight AI models with commercially exploitable licenses, verified benchmarks, and no geographic restrictions. Built to decide which models to support in herbert-rs, a local LLM inference engine in Rust and hand-written assembly.

Selection criteria:

  1. Commercially exploitable license, no geographic restriction (EU ok)
  2. VRAM Q4 ≤ 128 GB (main) — what fits Q4-quantized on a single high-end workstation. An extended tier (128 < Q4 ≤ 256 GB) flags models that require a Mac Studio M3 Ultra or multi-GPU server.
  3. Released after April 2024

This excludes Llama 4 multimodal (EU exclusion), Qwen 3.6 Plus (closed-source), DeepSeek V3/R1 full (671B → ~370 GB Q4, beyond 256 GB), and others. Note: Llama text-only models (3.3 70B, 3.2 1B/3B) are EU-exploitable. See Rejected models for details.

Maintained by Philippe Anel. Last updated: October 2026.

Tracking new releases. ROOTS.md lists every root model (published by the lab that trained it — no third-party fine-tunes, quantizations or merges) released by the followed organisations, newest first, regenerated by tools/hf_roots.py from Hugging Face. It is the candidate pool for this curated list. See Tracking new releases.

v5 additions (October 2026) — First batch from the hf_roots root-model scan, root models only. Generalists: Qwen3.8-27B (dense hybrid DeltaNet, GPQA 89.2), Muse-Glimmer-30B (Meta, SWE-bench V 76.0), Kolibri-1 78B-A3.5B (Aleph Alpha, German/English from scratch), Raon-1-mini 29B-A4B (Krafton), Nemotron 3.5 Lightning 30B-A3B + Super-VL 120B-A12B (NVIDIA), Ling-3.0-flash 124B-A5B (Ant), AliceAI-Foundation 80B-A3B (Yandex, base). Code: Laguna XS 2.1 / S 2.1 / M.1 (Poolside), North-Mini-Code (Cohere). Compact/Edge: MiniCPM5-2B, Ling-3.0-tiny, LFM3-1.7B. Alternative architectures: DiffusionGemma 26B-A4B (discrete text diffusion), Qwen3.8-Flash-Next (Qwen4 architecture preview). Extended tier: GLM-5.3-Flash, Hy3, Inkling-Small, MiMo-V2.6-Flash, Motif-3, GigaChat 3.5 Ultra, Solar Open 2, MiniMax-M3.1-Flash. Kept exception (derivative on purpose): Nemotron 3.5 Bolt — 74B-A9B compressed from Super-VL, E4B fine-tuned from Gemma 4 E4B. New license row: OpenMDW-1.1. Rejected: the new > 256 GB frontier wave (DeepSeek-V4.1-Flash, Kimi-K3, GLM-5.3, Mistral Large 4, Muse Spark Open…). Left out as undeclared derivatives of another lab's model: MiniCPM-V 4.7 (Qwen3.5 backbone), Mage-VL (Qwen3-4B), Unlimited-OCR (DeepSeek-OCR).

v4 additions (June 2026) — Generalists: Gemma 4 12B Unified (encoder-free multimodal, AIME 2026 77.5 / LCB v6 72.0). Code (LCB/Terminal-Bench): Mellum 2 12B-A2.5B (JetBrains, LCB v6 69.9 with 2.5B active). Compact/Edge: MiniCPM-V 4.6 (1.2B mobile-first MLLM). Vision/Multimodal: Lance (ByteDance, unified any-to-any 3B, gen + understanding + editing). Alternative architectures: HRM-Text-1B (Sapient, Hierarchical Reasoning Model, recurrent dual-timescale). New section: Web world models with WebWorld 8B/14B/32B (Qwen, predicts next page state, beats GPT-4o on Factuality).

v3 additions (May 2026) — Selection criterion switched from "< 200B params" to "VRAM Q4 ≤ 128 GB" + extended 256 GB tier (better proxy for what's actually runnable on consumer/prosumer hardware). Generalists: Ling-2.6-flash 104B/A7.4B (Ant), Mistral Medium 3.5 128B (🔴 Modified MIT), MiniMax M2.7 230B/A10B. Extended tier: DeepSeek-V4-Flash 284B/A13B (1M ctx native, FP4+FP8). Alternative architectures: ZAYA1-8B (Zyphra), Kimi-Linear 48B/A3B (Moonshot, KDA hybrid), Bonsai-8B (1-bit end-to-end, 1.15 GB). Vision/Multimodal: Nemotron 3 Nano Omni 30B-A3B, DeepSeek-OCR + DeepSeek-OCR-2. Compact/Edge: LFM2.5-VL-450M (vision edge). Theorem provers: SGS algorithm (Stanford, 7B beats 671B pass@4 on D3k). New license row: 🔴 Modified MIT with explicit warning.

v2 additions (April 2026) — Generalists: GLM-4.7-Flash, Hermes 4-70B. Code: NousCoder-14B, OmniCoder-9B (new LCB/Terminal-Bench subsection). Compact/Edge: Pleias-RAG-1B, Pleias-3B. Reasoning/Math: Qwen2.5-Math-72B (historical). Alternative architectures: URM. Decentralized training: Hermes 4.3-36B-Psyche. Theorem provers: Nomos 1 (natural-language track).


Table of Contents


LLMs

Generalists

ModelPublisherActiveTotalArchCtxLicenseKey scores
Gemma 4 31BGoogle31B31BDense256KApache 2.0GPQA 84.3, MMLU-Pro 85.2
Gemma 4 12B UnifiedGoogle11.95B11.95BDense encoder-free multimodal256KApache 2.0GPQA 78.8, MMLU-Pro 77.2, AIME 2026 77.5, LCB v6 72.0, audio natif (CoVoST 38.5)
Qwen3.5-27BAlibaba27B27BDense128KApache 2.0201 languages
Qwen3.5-9BAlibaba9B9BDense128KApache 2.0GPQA 81.7 (9B!)
Qwen3.5-122B-A10BAlibaba10B122BMoE256KApache 2.0201 languages, multimodal
GPT-OSS-120BOpenAI5.1B117BMoE128KApache 2.0GPQA 80.9, Codeforces 2622, AIME 96.6%
GPT-OSS-20BOpenAI3.6B21BMoE128KApache 2.0AIME 96%, fits 16GB
Mistral Small 4Mistral6B119BMoE256KApache 2.0GPQA 71.2, unified instruct/reasoning/coding
GLM-4.5-AirZhipu AI12B106BMoE128KMITMATH-500 98.1%, MMLU-Pro 81.4
GLM-4.7-FlashZhipu AI3B30BMoE (MLA)200KMITSWE-bench 59.2, AIME25 91.6, GPQA 75.2
Ling-2.6-flashAnt Group7.4B104BMoE (hybrid linear attn 1:7 MLA+Lightning)262KMITToken-efficient agent (~15M tokens on full AA suite vs 40-100M for long-reasoners)
QwQ-32BAlibaba32B32BDense128KApache 2.0AIME ~80%, reasoning RL
DeepSeek R1-Distill-32BDeepSeek32B32BDense128KMITBeats o1-mini
Step-3.5-FlashStepFun11B196BMoE262KApache 2.0SWE-bench 74.4%, 350 tok/s
Llama 3.3 70BMeta70B70BDense128KLlama Community (EU OK)MMLU 86.0, HumanEval 88.4, MATH 77.0
Hermes 4-70BNous Research70B70BDense128KLlama Community (EU OK)SOTA RefusalBench, hybrid reasoning, tool calling
InternVL3-78BShanghai AI Lab78B78BDense--Apache 2.0MMMU 72.2, SOTA open-source VLM
Mistral Medium 3.5 128BMistral AI128B128BDense + Pixtral vision256K🔴 Modified MIT (revenue cap)First Mistral merged flagship: Medium 3.1 + Magistral + Devstral 2 unified, configurable reasoning_effort
Qwen3.8-27BAlibaba27B27BDense hybrid (3:1 Gated DeltaNet / Gated Attention) + MTP, native VLM262K (1M YaRN)Apache 2.0GPQA 89.2, LCB v6 90.3, Terminal-Bench 2.1 73.0, SWE-bench Pro 61.7
Muse-Glimmer-30BMeta (MSL)29.6B29.6BDense, 3:1 local/global attn, ViT-G vision, DFlash drafter131KApache 2.0SWE-bench V 76.0, GPQA 83.5, AIME 2026 94.7, MMMU Pro 74
Kolibri-1Aleph Alpha3.46B78BMoE (384 experts, top-6), 4:1 SWA/global262K (1M extrapolated)Apache 2.0GPQA 84.8, MMLU-Pro 80.1, AIME 2026 96.7, LCB v6 85.1, RULER 1M 63.2 (base) — BF16 card
Raon-1-mini-previewKrafton4.3B28.8BHybrid MoE (Raon Delta Attention 3:1 NoPE MLA, mHC)262KApache 2.0 (gated)GPQA 84.3, MMLU-Pro 76.5, AIME 2026 93.3, LCB v6 81.6, SWE-bench V 70.8
Nemotron 3.5 Super-VLNVIDIA12B120BMamba-2/Transformer MoE + MTP, C-RADIO vision1MOpenMDW-1.1GPQA 89.52, LCB v6 88.22, SWE-bench V 71.10, Terminal-Bench 2.1 60.67
Nemotron 3.5 Bolt 74B-A9BNVIDIA9B74BMamba-2/Transformer hybrid LatentMoE, C-RADIO v4-H vision — compressed from Nemotron 3.5 Super-VL256KOpenMDW-1.1GPQA 88.83, Terminal-Bench 2.1 53.93, BrowseComp 70.00; "24GB" IQ2 build: GPQA 86.43
Nemotron 3.5 LightningNVIDIA3B30BMamba-2/MoE/attention hybrid + MTP, NVFP4 pre-training1MOpenMDW-1.1MMLU-Pro 81.94, GPQA 75.44, SWE-bench V 51.56
Ling-3.0-flashAnt Group5.1B124BMoE (512 experts, top-8), hybrid KDA 5:1 MLA + MTP256KMITCard claims parity with Ring-2.6-1T (scores published as images only)
AliceAI-Foundation-80B-A3BYandex3B80BMoE (512 experts, top-10), hybrid KDA 3:1 gated attn + MTP — base model262KApache 2.0Base 5-shot: MMLU-Pro 66.8, MATH-500 91.1; strong Russian (WikiWebFacts 86.5)
MiniMax M2.7MiniMax AI10B230BMoE (256 experts, 8 active, 4.3% ratio)~200KMIT (verify on HF)Agentic workflows alt to Claude Opus 4.6 / GPT-5.3-Codex, IQ1_M @ 60.7 GB

🔴 License warning — Modified MIT (revenue/MAU caps). Mistral Medium 3.5 falls under a Mistral Open License variant with a revenue threshold; MiniMax M2.7 and Kimi K2.5 historically shipped with similar caps (100M MAU for Kimi). They are listed for completeness but you must read the actual license before any commercial deployment — these are not interchangeable with Apache 2.0/MIT. The revenue/MAU clauses can flip a free model into a paid one once your product takes off.

Mistral Medium 3.5 (Apr/May 2026) is the first merged flagship from Mistral: a single set of weights unifying what used to be three distinct models — Medium 3.1 (instruct), Magistral (reasoning), Devstral 2 (coding agent). Behavior switches via reasoning_effort per request (none / high). Replaces Medium 3.1 + Magistral in Le Chat and Devstral 2 in Vibe CLI. 88-layer dense (no MoE), Pixtral vision tower trained from scratch.

Qwen3.8-27B (Aug 2026) keeps the 27B dense size of Qwen3.5/3.6 but switches to a hybrid linear-attention layout (three Gated DeltaNet layers per Gated Attention layer) with multi-step MTP and native vision. On paper it sits with the best MoEs ≤ 128B of the list on GPQA (89.2, vs 89.52 for Nemotron 3.5 Super-VL) and leads LiveCodeBench v6 (90.3) — dense is not dead at the 27B point.

Muse-Glimmer-30B (Meta Superintelligence Lab, Aug 2026) is an Apache 2.0 dense 30B distilled from Muse Spark (Meta's own flagship) for agentic work on consumer hardware — 4-bit fits 24/32 GB, and the shipped DFlash block-diffusion drafter gives 3.1× on an RTX 5090 per the card. Separate usage policy (no under-18 users).

Kolibri-1 (Aleph Alpha, Oct 2026) is a sovereign German/English MoE trained from scratch on 20T tokens with a German-optimized tokenizer; 3.46B active out of 78B, RoPE only in the sliding-window layers. One of the few 1M-context claims backed by a RULER 1M number (63.2, base model). Apache 2.0 grant scoped to the published weights and config files.

Nemotron 3.5 Super-VL is initialized from NVIDIA's own Nemotron 3 Super Base and distilled from Nemotron 3 Ultra Base — a first-party lineage, not a third-party fine-tune. NVIDIA's 3.5 generation moves from the Nemotron OML to OpenMDW-1.1 (permissive).

Nemotron 3.5 Bolt is kept as a deliberate exception to the root-models-only rule ([keep] in config.toml). Bolt 74B-A9B is Nemotron 3.5 Super-VL compressed by NVIDIA with Star Elastic (router-guided architecture search + distillation: expert top-K 22 → 9, 120B → 74B), sized to run below 24 GB at 128K context in its native 2-bit "24GB" build. Bolt-E4B (see Compact / Edge) is not an NVIDIA architecture at all: it is a post-training of Google's Gemma 4 E4B-it.

MiniMax M2.7 (Apr 2026 open-weight release) pushes the active/total ratio to 4.3% (10B/230B), targeting agentic long-running workflows (coding, multi-step troubleshooting, document editing). Positioned as open-weight alternative to Claude Opus 4.6 / GPT-5.3-Codex with IQ1_M weights at 60.7 GB making 230B practically deployable on a single workstation.

Extended tier (128 < Q4 ≤ 256 GB)

Models that exceed the 128 GB Q4 main cap but fit a 256 GB workstation (Mac Studio M3 Ultra max, multi-GPU server).

ModelPublisherActiveTotalArchCtxLicenseKey scores
DeepSeek-V4-FlashDeepSeek13B284BMoE (hybrid CSA+HCA, mHC, Muon optimizer)1M nativeMITFirst < 200B-active LLM with native 1M ctx, FP4+FP8 mixed, 32T pre-train, 27% FLOPs / 10% KV cache vs V3.2
Inkling-SmallThinking Machines12B276BMoE (256 experts, 6+2 shared), 5:1 local/global, text+image+audio in1M (config)Apache 2.0SWE-bench V 80.2, Terminal-Bench 2.1 64.7, GPQA 89.5, AIME 2026 95.5, MMMU Pro 74.0
GLM-5.3-FlashZhipu AI18B320BNative multimodal MoE, hybrid KDA + sparse attention, mHC, MTP1M (config)MITNew base (30T multimodal tokens); claims > GLM-5.2 at 1/10 the price (scores as images)
Hy3Tencent21B295BMoE (192 experts, top-8) + MTP256KApache 2.0Agentic/productivity focus; Tencent flagship under Apache 2.0 (no EU exclusion in the card, unlike earlier Hunyuan community licenses)
MiMo-V2.6-FlashXiaomi15B309BMoE (256 experts, top-8), 39 SWA / 9 global, MTP drafter, text+image+video+audio in1MMITTerminal-Bench 2.1 87.6, OSWorld-Verified 80.8, DeepSWE v1.1 67.9
Motif-3Motif Technologies13.2B314BMoE (384 experts, top-8), GDLA attention (differential + latent KV), mHC, MTP256KMITSWE-bench V 76.2, Terminal-Bench 2.1 74.9, GPQA 83.4 — from scratch, EN/KO
GigaChat 3.5 UltraSber28B432BMoE, hybrid MLA + GatedDeltaNet, 2 MTP heads, native FP8262K (config)MITMMLU-Pro 75.68, SWE-bench V 42.6, ~4× less KV cache than 3.1 Ultra — Q4 ≈ 238 GB, at the edge of the tier
Solar Open 2Upstage15B250BMoE (320+1 experts), 3:1 linear (KDA) / softmax, NoPE1M🔴 Upstage Solar License (naming + attribution)GPQA 86.3, AIME 2026 95.7, LCB v6 92.4, SWE-bench V 70.4
MiniMax-M3.1-FlashMiniMax23B~428BNative multimodal MoE, sparse attention in every layer, QAT (MXFP8), DSpark drafter1M🔴 minimax-community (read before use)DeepSWE v1.1 73.7, HLE 43.4 (no tools) — Q4 ≈ 235 GB

Hybrid linear attention is now the default above 100B. GLM-5.3-Flash (KDA + sparse), Solar Open 2 (KDA, only 12/48 layers hold a KV cache), GigaChat 3.5 (MLA + GatedDeltaNet), plus Ling-3.0, AliceAI and Raon in the main tier: all mix a linear-attention family (Kimi Delta Attention / Gated DeltaNet) with a minority of full-attention layers — the pattern Kimi-Linear demonstrated in Oct 2025 (see Alternative architectures).

DeepSeek-V4-Flash (May 2026) is the small sibling of V4-Pro (1.6T/49B). Q4 ≈ 156 GB. Three inference modes integrated in the chat template: non-think, think-high, think-max (recommended at ≥ 384K context). The architecture introduces three new ideas — hybrid attention (CSA + HCA), multi-head computation (mHC), and the Muon optimizer — pushing the efficiency frontier rather than the parameter frontier.

Code

ModelSWE-benchCodeforcesActiveLicense
Claude Opus 4.6 (closed)80.8%------
Gemini 3.1 Pro (closed)80.6%------
GPT-5.4 (closed)~80%------
Inkling-Small (extended tier)80.2%--12BApache 2.0
Motif-3 (extended tier)76.2%--13.2BMIT
Muse-Glimmer-30B76.0%--29.6BApache 2.0
Laguna M.174.6%--23BApache 2.0
Step-3.5-Flash74.4%--11BApache 2.0
Nemotron 3.5 Super-VL71.1%--12BOpenMDW-1.1
Laguna XS 2.170.9%--3BOpenMDW-1.1
Raon-1-mini-preview70.8%--4.3BApache 2.0
Kolibri-169.2%--3.46BApache 2.0
Devstral 272.2%--~12BMIT modified
Qwen3-Coder-Next 80B-A3B70.6%--3BApache 2.0
Qwen2.5-Coder-32B69.6%--32BApache 2.0
Devstral Small 268.0%--24BApache 2.0
GLM-4.7-Flash59.2%--3BMIT
GPT-OSS-120B62.4%26225.1BApache 2.0
Gemma 4 31B--215031BApache 2.0
MiniCPM5-2B46.4%--2.5BApache 2.0

SWE-bench = real bugs in real GitHub repos (Django, Flask, scikit-learn). 500 human-validated issues. Codeforces = algorithmic competition, ELO-scored like chess. Different skills: fixing a codebase vs solving a puzzle.

Code — LiveCodeBench & Terminal-Bench

Specialized coders measured on benchmarks other than SWE-bench.

ModelLiveCodeBench v6Terminal-BenchActiveLicense
Qwen3.8-27B90.373.0 (2.1)27BApache 2.0
Nemotron 3.5 Super-VL88.2260.67 (2.1)12BOpenMDW-1.1
Kolibri-185.1--3.46BApache 2.0
Raon-1-mini-preview81.6--4.3BApache 2.0
Laguna S 2.1--70.2% (2.1)~8B (118B total)OpenMDW-1.1
Muse-Glimmer-30B--51.7 (2.1, terminus2)29.6BApache 2.0
Laguna M.1--45.8% (2.0)23BApache 2.0
Laguna XS 2.1--37.5% (2.0)3BOpenMDW-1.1
MiniCPM5-2B69.1--2.5BApache 2.0
North-Mini-Code-1.0(image only)(image only)3B (30B total)Apache 2.0
OmniCoder-9B--23.6% (2.0)9.4BApache 2.0
NousCoder-14B67.87%--14.8BApache 2.0
Mellum 2 12B-A2.5B69.9--2.5B (MoE 64 experts top-8)Apache 2.0
Qwen3.5-9B (baseline)60.79%14.6% (2.0)9BApache 2.0

LiveCodeBench (rotating ≈700 problems from LeetCode/AtCoder/Codeforces, collected after model cutoffs) measures fresh competitive programming, vs SWE-bench (fixing real-world bugs) and Codeforces ELO (pure algorithms). Terminal-Bench 2.0 measures agentic coding skills (read-before-write, LSP responsiveness, minimal diffs).

OmniCoder-9B is a LoRA agentic fine-tune of Qwen3.5-9B on 425K Claude Opus 4.6 / GPT-5.4 / Gemini 3.1 Pro trajectories — +61% relative on Terminal-Bench vs base. NousCoder-14B is a pure-RL fine-tune of Qwen3-14B (+7.08 pts on LCB v6, no SFT). Same 9-14B class, opposite methods.

Poolside Laguna (Jun–Jul 2026) is a three-size agentic-coding family trained by Poolside: XS 2.1 (33B-A3B, runs on a 36 GB Mac), S 2.1 (118B-A8B, 1M context, 1:3 global/SWA) and M.1 (225B-A23B, global attention everywhere, Q4 ≈ 124 GB — the top of the main tier). XS and S ship under OpenMDW-1.1, M.1 under Apache 2.0; all three are gated (privacy notice) and come with DFlash drafters. North-Mini-Code-1.0 (Cohere, 30B-A3B, Apache 2.0, 256K) is Cohere's open agentic coder; its benchmark table is only published as an image.

Terminal-Bench versions (2.0 / 2.1) are not comparable; the version is given in each cell.

Mellum 2 (JetBrains, May 2026) is the second generation after Mellum 1 (4B dense, 2024) — this time a MoE 12B/A2.5B post-trained in SFT + RLVR (Reinforcement Learning with Verifiable Rewards). JetBrains publishes 6 checkpoints exposing each pipeline stage (Base-Pretrain, Base, Instruct-SFT, Thinking-SFT, Instruct, Thinking) for research transparency — rare among IDE vendors. LCB v6 69.9 with 2.5B active is exceptional in the 9-14B class; the active-param efficiency makes it a natural fit for in-IDE inference.

Reasoning

GPQA Diamond (198 questions)

Graduate-level questions in physics, chemistry, biology. Designed to be unsolvable by Google search. Experts reach 65%, non-experts 34%. The most discriminating reasoning benchmark available.

ModelGPQAActive
Gemini 3.1 Pro (closed)94.3--
GPT-5.4 (closed)92.8--
Claude Opus 4.6 (closed)91.3--
Nemotron 3.5 Super-VL89.5212B
Inkling-Small (extended)89.512B
Qwen3.8-27B89.227B
Nemotron 3.5 Bolt 74B-A9B88.839B
Kolibri-184.83.46B
Gemma 4 31B84.331B
Raon-1-mini-preview84.34.3B
Muse-Glimmer-30B83.5 (AA)29.6B
Gemma 4 26B-A4B82.33.8B
Qwen3.5-9B81.79B
GPT-OSS-120B80.95.1B
GLM-4.5-Air75.012B
Nemotron 3 Nano73.03.5B
Mistral Small 471.26B
MiniCPM5-2B70.2 (AA)2.5B
Llama 3.3 70B50.570B

Math (AIME, 15 problems/year)

Competition-level math requiring creativity and multi-step reasoning. Each year's edition is different and harder. Only compare within the same version.

ModelAIMEConditionsActive
GPT-5.4 (closed)~100%2025--
Claude Opus 4.6 (closed)~98%2025--
Nemotron 3 Nano99.2%2025, with tools3.5B
GPT-OSS-120B96.6%2024, with tools5.1B
GPT-OSS-20B96.0%2024, with tools3.6B
Kolibri-196.7%2026, reasoning high3.46B
Inkling-Small (extended)95.5%202612B
Muse-Glimmer-30B94.7%2026, high reasoning29.6B
Raon-1-mini-preview93.3%20264.3B
Gemma 4 31B89.2%202631B
Gemma 4 26B-A4B88.3%20263.8B
MiniCPM5-2B86.5%20262.5B
DiffusionGemma 26B-A4B69.1%2026, no tools, diffusion3.8B
Ministral 14B85.0%202514B
Nemotron Nano 9B v297.8%MATH-500, /think mode9B
Qwen2.5-Math-72B40.0%2024, TIR (Python)72B

AIME versions (2024/2025/2026) are not comparable. Each year is harder.

Qwen2.5-Math (Sep 2024) is the first open model to make real AIME progress (12/30 on AIME 2024 — 6× GPT-4 Turbo at the time). Since surpassed by generalists (Nemotron, GPT-OSS 96%+). Historical reference, still useful for its TIR mode (Tool-Integrated Reasoning with Python interpreter) which eliminates arithmetic errors. Qwen License, not Apache — commercial OK but check terms.

Compact / Edge

Models that run on smartphones, laptops, or edge devices.

ModelActiveVRAM Q4StrengthLicense
Nemotron 3.5 Bolt-E4B4B effective (8B)~4 GB (Q4_K_M "4GB" build)NVIDIA fine-tune of Gemma 4 E4B-it for voice/device assistants: IFBench 63.79 (Gemma 4 E4B: 42.84), RoleBench 67.05, τ²-Bench 55.10, 64K ctxOpenMDW-1.1 (base under Apache 2.0)
MiniCPM5-2B2.5B~1.5 GBAIME 2026 86.5, LCB v6 69.1, SWE-bench V 46.4, 128K ctx, training data released (UltraData)Apache 2.0
LFM3-1.7B1.7B~1 GBConv + SWA + global tri-hybrid, BFCL v4 71.4, LCB v6 73.5, SWE-bench V 43.4 (OpenCode), 256K ctxLFM Open v1.0
Ling-3.0-tiny1.3B (7.9B total)~4.5 GBHybrid KDA/MLA MoE for DGX Spark / Apple Silicon, AA Intelligence Index 25MIT
SmolLM3-3B3B~2 GBBest 3B, AIME 36.7%, /think mode, 64K ctxApache 2.0
SmolLM2-1.7B1.7B~1 GB11T tokens, data-centricApache 2.0
SmolLM2-360M360M< 1 GB4T tokensApache 2.0
SmolLM2-135M135M< 1 GBUltra-compact, few MB quantizedApache 2.0
Gemma 4 E2B2.3B~4 GBMultimodal + audioApache 2.0
Gemma 4 E4B4.5B~6 GBMultimodal + audioApache 2.0
Phi-4-mini3.8B~2 GBMATH-500 92.5%MIT
Phi-4-multimodal5.6B~3 GBText + image + audioMIT
Ministral 3B3B~2 GBVision + reasoning, 256K ctxApache 2.0
Ministral 8B8B~5 GBAIME 78.7%, visionApache 2.0
Ministral 14B14B~8 GBAIME 85%, vision, 256K ctxApache 2.0
LFM2.5-1.2B1.2B~1 GBIFBench 47.3 (2x Qwen3-1.7B), thinking, vision, audioLFM Open v1.0
Llama 3.2 1B/3B1-3B< 2 GB128K ctx, edge/mobile, EU OK (text-only)Llama Community
InternLM3-8B8B~5 GBThinking mode, 4T tokens (75% less training)Apache 2.0
InternVL3-1B→38B1-38B1-20 GBVision SOTA, full range edge→serverApache 2.0
Chocolatine-2-4B-DPO4B~2.5 GBFrench-optimized DPO fine-tune of Qwen3-4B, 262K ctx, no <think>Apache 2.0
Pleias-RAG-1B1.2B~1 GB100% public-domain training data, native citation with literal quotes, EU multilingualApache 2.0
Pleias-RAG-350M350M< 1 GBSame as Pleias-RAG-1B, ultra-compactApache 2.0
Baguettotron0.3B< 1 GBLatest Pleias base (Dec 2025), French-focused SLMApache 2.0
LFM2.5-VL-450M450M< 1 GBVision edge: SigLIP2 + 512×512 native, object detection, WebGPULFM Open v1.0
MiniCPM-V 4.61.2B< 1 GBMobile-first MLLM (Qwen3.5-0.8B + SigLIP2-400M), text + image + video, iOS/Android/HarmonyOS native patchesApache 2.0
Bonsai-8B8B1.15 GB1-bit Qwen3-8B fine-tune, CUDA/Metal/CPU/Android/iPhoneApache 2.0

SmolLM3-3B beats all other 3B models and competes with 4B models (Qwen3-4B, Gemma3-4B). Data quality matters more than model size: SmolLM2-1.7B trained on 11T tokens beats larger models trained on less data.

Chocolatine-2-4B (Jonathan Pacifico) is a DPO fine-tune of Qwen3-4B-Instruct-2507 on French preference datasets (Compar:IA from the French Ministry of Culture + French-ORCA), merged with TIES. Gains on every French benchmark tested (GPQA-FR, French MMLU, French Bench, FR-MT-Bench) without degrading English performance. One of the rare French-focused open-weight models built by an individual contributor rather than a lab.

Pleias (Paris-based lab, partners with NVIDIA and Mozilla Builders) trains exclusively on public-domain or CC-licensed data (Common Corpus, 2T tokens). Raw benchmark scores trail Qwen/Gemma of equivalent size, but the trade-off is unique: zero copyright ambiguity (EU AI Act / GDPR friendly), strong EU multilingual (FR, DE, IT, ES, NL, PL), and the Pleias-RAG variants emit literal-quote citations natively. Positioned for regulated sectors (public, legal, press, education) where data provenance matters more than peak scores.

Long context

ModelMax ctxRULER 1MArchitectureActiveLicense
Nemotron 3 Nano1M86.3%Mamba/MoE3.5BNemotron OML
Nemotron 3 Super1M--Mamba/MoE12BNemotron OML
DeepSeek-V4-Flash (extended tier)1M native--MoE (CSA+HCA hybrid)13BMIT
Jamba 1.6 Mini256K--SSM+Transformer/MoE12BJamba OML
Kolibri-11M (extrapolated from 262K)63.2% (base)MoE, 4:1 SWA/global, RoPE only in SWA3.46BApache 2.0
Nemotron 3.5 Super-VL1M--Mamba-2/Transformer MoE12BOpenMDW-1.1
Laguna S 2.11M--MoE, 1:3 global/SWA (window 512)~8BOpenMDW-1.1

RULER (GitHub) tests retrieval in long contexts with multiple needles, multi-hop tracing, and aggregation. Parametric by length (4K to 1M). Many models claim "1M context" without publishing RULER scores at that length. Without measurement, it's marketing.

Alternative architectures

Non-Transformer or hybrid models.

ModelArchitectureActiveKey metricLicense
Granite 4.090% Mamba-2 / 10% Attention3-9B70% memory reduction, 2x speedApache 2.0
LFM2/2.5Convolutions + grouped attention2.3B112 tok/s CPU, 2x Qwen3. LFM2.5: vision, audio, thinkingLFM Open v1.0
Jamba 1.6 MiniMamba + Transformer + MoE12B2.5x Transformer speedJamba OML
URMRecursive Universal Transformer (ConvSwiGLU + TBPTL)4× params (tiny)ARC-AGI 1: 53.8%, Sudoku 77.6%Open-source (research)
ZAYA1-8BHybrid Mamba + Compressed Cross Attention (CCA) + MoD + EDA760M / 8.4B (9% active)On-device deployable, test-time-compute friendly, 128K ctxApache 2.0
Kimi-Linear-48B-A3BMoE hybrid: 3 KDA (linear) layers per 1 MLA (global)3B / 48B1M context, 5.7T tokens, demonstrates linear attention can match full attentionMIT
Bonsai-8BQwen3-8B fine-tuned at 1-bit end-to-end (GGUF Q1_0), all projections + LM head 1-bit8.19B1.15 GB on disk (14.2× FP16), runs on CPU/Android/iPhoneApache 2.0
DiffusionGemma 26B-A4BDiscrete text diffusion on the Gemma 4 26B-A4B MoE: AR encoder for the prompt, bidirectional decoder denoising 256-token canvases3.8B / 25.2B15-20 tokens per forward, > 1100 tok/s on H100 FP8; MMLU-Pro 77.6 vs 82.6 for AR Gemma 4 26B-A4BApache 2.0
Qwen3.8-Flash-NextQwen4 architecture preview: Gated DeltaNet 3:1 Qwen Sparse Attention, Gated Residual, 51B offloadable n-gram embedding table6B / 125B (+51B n-gram, +4B MTP)GPQA 91.7, LCB v6 91.9, SWE-bench Pro 62.5, 262K (1M YaRN)🔴 qwen-community-1.0 (read before use)
HRM-Text-1BHierarchical Reasoning Model — dual-timescale recurrent (H/L cycles), PrefixLM objective~1B (16 layers per stack)Research base model (pre-alignment, no SFT/RLHF), 4K ctxApache 2.0

URM (Ubiquant, Dec 2025) loops its 4 layers 12× instead of stacking 48 distinct layers. With 4× parameters it reaches 53.8% on ARC-AGI 1 where a vanilla Transformer with 32× parameters stays under 40%. Key claim of the paper: the FFN, not attention, is the source of reasoning — counterintuitive given the community's focus on attention variants. Research model, not a production LLM, but architecturally interesting for future LLM designs. See arXiv:2512.14693.

ZAYA1-8B (Zyphra, May 2026) is a Zamba-2 successor: 80 layers mixing SSM-Mamba and attention, with Compressed Cross Attention (CCA) plus Mixture-of-Depths (MoD) and Expert Decision Attention (EDA) on top of 16 top-1 experts. The angle is intelligence per active parameter: 760M actifs gives sub-1B inference cost while keeping 8B-class capacity. Positioned for on-device + thinking-mode workflows where compute scales with active params, not totals. Tech report on zyphra.com/zaya1-8b-technical-report.

Kimi-Linear (Moonshot, Oct 2025, arXiv:2510.26692) is Moonshot's open research vehicle for linear attention outside the closed K2 family. The architecture is a 3:1 ratio of KDA (Kimi Delta Attention, linear) to MLA (full attention, global) layers. The point isn't frontier performance — it's the demonstration that linear attention can match full attention across short, long, and RL-style regimes while reducing memory cost. Useful baseline for engine work like herbert-rs.

Bonsai-8B (Prism ML, Mar 2026) is a 1-bit end-to-end fine-tune of Qwen3-8B: every projection + the LM head quantized to 1 bit (GGUF Q1_0), shrinking the deployed model to 1.15 GB. Direct competitor to BitNet, but trained as a fine-tune rather than natively 1.58-bit from scratch. Runs on CUDA, Metal, Android, CPU, and iPhone (via Locally AI). The radical end of the quantization spectrum — accept the quality drop in exchange for ubiquity.

DiffusionGemma (Google DeepMind, Jun 2026) is Google's open-weight diffusion LLM: same MoE as Gemma 4 26B-A4B, but tokens are produced by denoising a 256-token canvas in parallel, block after block. The card publishes the trade-off side by side: ~5 points lower on MMLU-Pro, ~19 lower on AIME 2026, for an order-of-magnitude higher per-user decode speed. For an inference engine, a completely different hot loop: bidirectional attention over the canvas, many tokens per forward.

Qwen3.8-Flash-Next (Aug 2026) is labelled by Qwen as the experimental preview of the architecture that will underpin Qwen4. Two engine-relevant novelties: Qwen Sparse Attention (micro-block budget of 2048 tokens) replacing Gated Attention, and a 51B n-gram embedding table (bigram/trigram lookups at layer 2) designed to be offloaded from GPU memory. Custom license, unlike the Apache 2.0 Qwen3.8-27B.

HRM-Text-1B (Sapient Intelligence, May 2026, arXiv:2605.20613) is the first language model built on the Hierarchical Reasoning Model architecture — a dual-timescale recurrent design that loops H (high-level/slow) and L (low-level/fast) cycles, similar in spirit to URM's recursive depth. Trained from scratch with PrefixLM objective, no SFT/RLHF/instruction tuning — it's a research base model, prompt few-shot to use. Context is only 4K, English only. Listed as a research vehicle for the community to experiment with adaptive-compute recurrent architectures, not a production LLM.

Decentralized training

Models pre-trained outside traditional data centers, using distributed peer-to-peer or blockchain-coordinated networks. The story is the training method, not the model quality.

ModelMethodSizeTokensArchitectureLicense
Covenant-72BPermissionless P2P, SparseLoCo optimizer, Bittensor blockchain (Subnet 3)72B dense1.1T (+14.8B SFT)LLaMA-3 style, GQA, 80 layers, d=8192, 64 heads, 8 KV heads, RoPE 500K, ctx 2048→8192Apache 2.0 (checkpoints)
Hermes 4.3-36B-PsycheInternet-decentralized fine-tuning via Psyche36B dense— (post-training on Seed-36B)ByteDance Seed-36B base, Llama-3 chat template, hybrid <think> modeApache 2.0

Pre-training benchmarks (0-shot) vs other dense baselines :

BenchmarkCovenant-72BLLaMA-2-70B (centralized)LLM360 K2 (65B, centralized)INTELLECT-1 (10B, P2P)
ARC-Challenge56.857.453.844.8
ARC-Easy80.979.676.071.8
PIQA81.682.682.577.4
OpenBookQA44.049.448.043.8
HellaSwag80.684.382.970.3
WinoGrande75.980.476.463.3
MMLU67.165.665.532.7

Covenant-72B-Chat (post-SFT) vs other chat models :

BenchmarkCovenant-72B-ChatLLaMA-2-70B-ChatK2-Chat (65B)
ARC-Challenge64.265.462.0
MMLU67.463.167.9
IFEval64.740.745.5
MATH26.310.719.1
MMLU-Pro40.935.245.4
GSM8K63.952.279.0

Hermes 4.3-36B-Psyche (Nous Research, Nov 2025) is a different point in the same space: not pre-training from scratch but post-training decentralized over internet. Built on ByteDance's Seed-36B, fine-tuned via Nous's Psyche network, released under Apache 2.0. The Psyche variant matches or beats the centralized 4.3-36B twin on every benchmark (AIME25 69.3 vs 66.8, MMLU-Pro 80.7 vs 79.7) — decentralized post-training did not degrade quality. Complements Covenant: two different decentralization angles (pre-training at 72B / post-training at 36B).

Why Covenant matters: Covenant-72B is the first proof-of-concept that 72B-scale pre-training is possible without data centers, with peers joining and leaving freely. Coordination via the Bittensor blockchain (Subnet 3), communication via SparseLoCo (146× compression vs dense gradients), peers running 8×B200 GPUs over commodity internet (500 Mb/s down, 110 Mb/s up). The model achieves 94.5% compute utilization despite the network constraints, with an average of 16.9 contributing peers per round and 70+ unique peers over the run. On benchmarks, it beats LLaMA-2-70B on ARC-Challenge, ARC-Easy and MMLU (despite 1.8× fewer training tokens), and the chat variant has the best IFEval and MATH scores in its comparison group. It's the first credible alternative to the data-center duopoly for pre-training at 70B scale. Authors: Covenant AI + Mila. See arXiv 2603.08163.


Specialized

Theorem provers (Lean 4)

miniF2F (GitHub): 488 formal Olympiad-level math problems. Proofs are compiler-verified: either correct or rejected. Zero hallucination possible on mathematical correctness.

ModelminiF2FPutnamBenchActiveLicense
BFS-Prover-V2-32B95.0%--32BApache 2.0
Goedel-Prover-V2-32B90.4%#132BApache 2.0
DeepSeek-Prover-V2-7B88.9%--7BMIT
DeepSeek-Prover-V2-7B + SGS----7BMIT (model) / CC-BY-4.0 (paper)
Leanstral----32BApache 2.0
Kimina-Prover-72B84.0%--72BMIT
Leanabell-Prover-V2-7B78.2%--7BApache 2.0

Lean 4 proofs are verified by the compiler. Either correct or rejected. Zero hallucination on mathematical correctness.

The sweet spot is 32B: BFS-Prover (95%) and Goedel-V2 (90.4%) both beat the 72B Kimina (84%).

SGS — Self-Guided Self-Play (Stanford, arXiv:2604.20209, Apr 2026) is not a model but an RL self-play algorithm applied to DeepSeek-Prover-V2-7B. After 200 rounds and 6.3M generations, the 7B fine-tune surpasses the pass@4 of DeepSeek-Prover-V2-671B on D3k (3 323 Lean 4 problems from Goedel-Pset-V1). Caveat: D3k is the SGS run's own training-target set, not a held-out public benchmark like miniF2F or PutnamBench — the 7B-beats-671B headline is real for in-distribution problems, scope-restricted otherwise. Demonstrates that well-tuned RL can collapse a 100× parameter gap on a target dataset. Authors: Bailey, Wen, Dong, Hashimoto, Ma.

Natural-language provers (not Lean 4)

A parallel track: models that write proofs in natural English, not formal Lean 4. Not compiler-verified, but closer to how mathematicians actually work.

ModelBenchmarkActiveTotalLicense
Nomos 1Putnam 2025: 87/120 (72.5%) with Nomos Harness~3B~30BApache 2.0

Nomos 1 (Nous Research × Hillclimb AI, Dec 2025) is a Qwen3-30B-A3B-Thinking fine-tune specialized for natural-language proof writing, not Lean 4. On Putnam 2025 with the open-sourced Nomos Reasoning Harness, it jumps from 24/120 (base) to 87/120 — a +63 point gain where the inference harness matters as much as the model. Complementary to the Lean provers above, which offer compiler-verification guarantees that natural-language proofs cannot.

GUI agents

ScreenSpot (GitHub): 1,200+ instructions across desktop, mobile, web. Tests if the model can locate the right UI element from a natural language instruction.

ModelScreenSpotOSWorldActiveLicense
UI-TARS-1.5-7B94.2%42.57BApache 2.0
Qwen2.5-VL-7B84.7--7BApache 2.0
ShowUI-2B----2BMIT

UI-TARS-7B beats Claude (87.6%) on ScreenSpot. 7B, Apache 2.0, runs on a laptop.

Web world models

A different category from web agents. Agent: decides "which action?". World model: predicts "what state comes next if I take this action?". The world model trains agents and enables inference-time lookahead — it does not pilot the browser itself.

ModelTypeSizesKey claimLicense
WebWorld-8B/14B/32BDense Qwen3-based, predicts next page HTML/A11y/Markdown8B / 14B / 32BBeats GPT-4o on Factuality (70-71 vs 59), parity Claude Opus 4.1 / Gemini 3 Pro; +10 pts MiniWob++/WebArena when used to generate agent training data; cross-domain +0.38 GUI DesktopApache 2.0

WebWorld (Qwen Team, May 2026, arXiv:2602.14721) is the first serious open-source web world model. Trained on 1M+ trajectories (Qwen/WebWorldData released separately). Two evaluation axes: Factuality (is the predicted next state causally correct?) and Web Turing (is it perceptually distinguishable from a real page?). The cross-domain transfer to GUI Desktop (+0.38) suggests the state → action → new state pattern generalizes beyond the web. Pair it with a decisional agent (UI-TARS, ShowUI, CogAgent) — WebWorld does not decide, it simulates.

Search agents

ModelSpecialtyActiveLicense
WebThinker-32BRL web search, beats Gemini Deep Research32BApache 2.0
DeepResearcher-7BEmergent multi-step planning via RL7BApache 2.0
Search-R1Framework: teach any LLM to search (+26% on 7B)anyApache 2.0

Tool calling

BFCL (GitHub): Berkeley Function Calling Leaderboard. Tests function/tool calling accuracy: correct names, parameters, types. V4 adds web search and memory.

ModelBFCLActiveLicense
Hammer2.1-7B#17BCC-BY-NC 4.0
xLAM-8B#1 (alternate)8BCC-BY-NC 4.0
Hammer-0.5BOn-device0.5BCC-BY-NC 4.0

Specialized tool-calling models clearly beat generalists. xLAM-8B beats GPT-4o on BFCL.

Rust

ModelStrandset-RustRustEvo2ActiveLicense
Strand-Rust-Coder-14B0.500.4314BApache 2.0 (base)

Beats GPT-5-Codex and Claude Sonnet 4.5 on Rust benchmarks. Fine-tuned on 191K examples from 2,383 crates.

Vision / Multimodal

ModelMMMUActiveKey featureLicense
InternVL3-78B72.278BSOTA open-source VLM, custom InternViTApache 2.0
InternVL3-1B→38B--1-38BFull range edge→serverApache 2.0
Gemma 4 31BPro 76.931BText + image + videoApache 2.0
Gemma 4 E2B/E4B--2.3-4.5BMultimodal + audio, edgeApache 2.0
Qwen2.5-VL-7B--7BComputer/phone use, DocVQA 95.7Apache 2.0
Nemotron 3 Nano Omni 30B-A3B--3Bany-to-any (text+audio+image+video → text), 256K ctx, Mamba-Transformer hybrid MoENemotron OML
DeepSeek-OCR--3.3BOCR specialist — Contexts Optical Compression: encode long text as compressed image, feed books/papers as pixels not tokensMIT
DeepSeek-OCR-2--3.4BOCR v2 with Visual Causal Flow (sequential reading order)Apache 2.0
Lance--3BUnified any-to-any (Qwen2.5-VL-3B base): image+video understanding + generation + editing in one model, 768×768 / 480p@12fpsApache 2.0

InternVL3-78B (72.2 MMMU) is on par with GPT-4o on multimodal. The InternViT encoder (300M–6B) is trained jointly with the LLM — not bolted on after the fact.

Nemotron 3 Nano Omni (NVIDIA, Apr 2026) extends the Nano family with native audio/video/image inputs. Targets enterprise document intelligence (contracts, SOW/MSA, finance), customer service (drive-thru order verification, delivery video OCR), GUI/browser/email agents, and dense video captioning. Stack of three components around the Nano LLM (vision encoder + audio encoder + LLM), not a monolithic any-to-any architecture. English-only. See arXiv:2604.24954.


Observations

Patterns observed across 90+ models. Not definitive truths.

Architecture

  • Dense retreats above 35B, but doesn't die. For generalists above 35B, MoE clearly dominates (GPT-OSS-120B, Mistral Small 4, Qwen3.5-122B-A10B, GLM-4.5-Air, Step-3.5-Flash, Nemotron 3 Super, all MoE). But dense survives where it has a structural advantage: Llama 3.3 70B (generalist), InternVL3-78B (vision), Kimina-Prover-72B (theorem proving), Qwen 2.5-72B (production NLP), Covenant-72B (decentralized training), DeepSeek R1-Distill-70B (distilled reasoning). Dense is becoming a specialization choice.

  • Parameter count is no longer the determining factor. Qwen3.5-9B (9B) beats GPT-OSS-120B (5.1B active, 117B total) on GPQA Diamond.

  • The 40-79B segment is the dense survivors' refuge. New models often jump from ~35B straight to ~120B total via MoE. But the 40-79B range is well populated by quality dense models (Llama 3.3 70B, InternVL3-78B, Kimina-Prover-72B, Qwen 2.5-72B, Covenant-72B, R1-Distill-70B, Jamba 1.6 Mini 52B). This is where dense resists, and where you find both solid generalists and specialists.

  • InternVL3 is the best open-source VLM nobody was talking about. InternVL3-78B (Shanghai AI Lab) reaches 72.2 MMMU under Apache 2.0 — on par with GPT-4o. InternLM3-8B achieves SOTA with 75% fewer training tokens (4T vs 15-18T). Less press than Alibaba, comparable results.

  • Undeclared derivatives are common, even from big labs. The hf_roots scan flags repos that declare no base_model but have exactly the parameter count of another lab's model: MiniCPM-V 4.7 (Qwen3.5/3.6 backbone, acknowledged only in the card), Mage-VL (Qwen3-4B), Unlimited-OCR (DeepSeek-OCR architecture), DeepSeek-R1-Distill, UI-TARS… A "new model" from a known lab is not automatically a new pre-training run.

  • Qwen is the de facto base model for fine-tuning. BFS-Prover, Goedel-Prover, Kimina-Prover, most community distillations: all built on Qwen. The ResNet of LLMs.

  • Decentralized pre-training is no longer a toy. Covenant-72B (Mar 2026) pre-trained a 72B dense LLaMA-3-style model over a permissionless blockchain network (Bittensor Subnet 3) on 1.1T tokens. It beats LLaMA-2-70B on ARC-Challenge, ARC-Easy and MMLU despite 1.8× fewer training tokens, with 94.5% compute utilization over commodity internet (500/110 Mb/s) and dynamic peer participation. The data-center duopoly for pre-training at 70B scale now has a credible alternative. SparseLoCo + 2-bit quantization gives 146× compression on gradient communication.

Benchmarks

  • GPQA Diamond is the most discriminating benchmark for reasoning: 198 doctoral-level questions, impossible to solve by retrieval.

  • SWE-bench vs Codeforces measure different things. GPT-OSS-120B dominates competition (ELO 2622) but gets beaten on real bugs by Step-3.5-Flash (74.4% vs 62.4%).

  • Many models claim "1M context" without RULER scores at that length. Without measurement, it's marketing.

  • AIME versions (2024/2025/2026) are not comparable. Each year is harder. Only compare within the same version.

Specialization

  • Specialized models dominate on narrow tasks. UI-TARS-7B beats Claude on GUI (94.2% vs 87.6%). BFS-Prover-32B beats DeepSeek-671B on theorem proving (95% vs 88.9%).

  • The sweet spot for theorem proving is 32B. Method (tree search, self-correction) compensates for size.

  • Domain-specific models (medical, legal, finance) are less mature than code/math specialists. Generalists often outperform them on domain benchmarks. Specialization helps mainly for specific vocabulary, regulatory compliance, and private data fine-tuning.

Inference acceleration

Three orthogonal axes accelerate inference on open-weight models. MTP reduces forward passes (build draft tokens that get verified in a single target pass). External drafters (EAGLE-3) do the same with a separate trained model, achieving higher absolute speedups but requiring more setup. KV cache quantization reduces memory per token, freeing room for longer context or larger batches — composes orthogonally with either of the spec-decoding paths.

MTP / speculative decoding (built-in)

Multi-Token Prediction (MTP) is becoming standard for open-weight MoE ≥ 100B in 2025-2026: a small extra prediction module is trained jointly with the main model and reused at inference as a built-in draft path for speculative decoding. No external drafter required. Among the models listed here:

ModelMTP at training?MTP at inference?Variant
DeepSeek-V4-Flash (extended)Yes (V3 lineage)YesSequential modules, λ schedule 0.3→0.1 in decay
Step-3.5-Flash 196BYesYes — MTP-3 active3 drafted tokens per forward
Nemotron 3 Super 120B-A12BYesYesShared-weight MTP head as internal draft
MiniMax M2.7 230BYes (K=1)Yes (K=3)Expanded K=1→K=3 via weight copying in decay (DeepSeek-V3 schedule)
Gemma 4 family (E2B / E4B / 12B Unified / 26B-A4B / 31B)NoYes — post-hoc drafterSeparate 4-layer Q-only model reading target's KV cache (gemma-4-X-it-assistant)

v5 note — drafters now ship next to the weights. Beyond MTP heads, most Oct 2026 releases publish a separate speculative-decoding drafter in the same org: DFlash block-diffusion drafters (Muse-Glimmer-30B: 3.1× on RTX 5090, Poolside Laguna, Nemotron 3.5 Lightning) and DSpark (MiniCPM5, Ling-3.0-flash, Nemotron 3.5, MiniMax-M3.1-Flash, DeepSeek-V4.1-Flash). The hf_roots tool folds them into their model's family.

Pattern. Two doctrines coexist:

  • MoE ≥ 100B: MTP shipped inside the model (DeepSeek-V3 / Step / Nemotron / MiniMax). MiniMax M2.7 replicates DeepSeek-V3's exact training schedule (λ=0.3→0.1) — convergence on hyperparameters across labs.
  • Gemma 4: MTP shipped as a separate post-hoc drafter model — same speedup ceiling (~1.5-3× lossless), but the drafter is an artifact you load independently.

Distribution layer — Unsloth. For practitioners, Unsloth (Daniel Han + team, Apache 2.0 tooling) packages MTP-enabled models as GGUF builds with their Dynamic 2.0 quantization — important layers upcast, real-world calibration. Their unsloth/Qwen3.6-27B-MTP-GGUF is the canonical example (1M+ downloads, 635 likes, "MTP enables ~1.5-2× faster inference with no accuracy loss"). Unsloth Studio auto-tunes MTP settings per hardware (Mac/CPU/GPU). Practical effect: MTP-enabled Qwen, Gemma 4, etc. become runnable on llama.cpp + consumer hardware without manual drafter wiring.

Why this matters for inference engine choice (herbert-rs et al.). A model that ships with MTP gives 1.5-3× throughput out of the box on supported frameworks (vLLM, SGLang, MLX, llama.cpp community) — no drafter to source, train, or quantization-match. Running an MTP-equipped model without spec-decode support is leaving 50-200% throughput on the table.

Sources. Per-model HF cards · DeepSeek-V3 paper · MiniMax-M2 tech report (arXiv:2605.26494 §2.3) · Gemma 4 Google blog · Unsloth Qwen3.6 MTP model card · Taxonomy: Sebastian Raschka LLM Architecture Gallery — MTP + Gloeckle et al. 2024 (arXiv:2404.19737, founding paper).

External drafters — EAGLE-3

For models that don't ship native MTP, the academic SOTA for speculative decoding is EAGLE-3 (Li, Wei, Zhang, Zhang — NeurIPS 2025, arXiv:2503.01840). The key innovation: abandons feature-level prediction in favor of direct token prediction, combined with multi-layer feature fusion through a technique called training-time test — letting the draft model fully benefit from scaling up training data.

Numbers. On Llama 3.1 70B AWQ INT4, single H100, batch=1, vLLM, with K=7 drafted tokens : 3.6× speedup vs vanilla decoding (per LocalAIMaster May 2026 bench). For comparison: EAGLE-2 reaches 3.0× and a generic draft-Llama-3.2-1B model reaches 2.2× in the same setup. EAGLE-3 beats MTP-based drafters in absolute speedup terms.

Trade-off vs MTP. EAGLE-3 wins on raw speed but loses on integration: you must source or train the draft model independently, tokenizer-match it, and quantization-match it to the target. MTP-equipped models trade ~0.5× speedup for zero setup. Above 70B without native MTP support, EAGLE-3 is currently the only path to 3×+ speedup.

Frameworks. TensorRT-LLM (production H100/H200), SGLang (community), vLLM (via custom config). Pre-trained EAGLE heads are available on Hugging Face for Llama 3.1 8B/70B (yuhuili/EAGLE-LLaMA3.1-Instruct-*) and a few other targets — but coverage is uneven, which is partly what Unsloth's MTP-packaging strategy is replacing for Qwen.

Sources. EAGLE-3 paper arXiv:2503.01840 (NeurIPS 2025) · Official repo SafeAILab/EAGLE · LocalAIMaster spec-decoding benchmark May 2026.

KV cache quantization — Google TurboQuant + community

The KV cache is the memory bottleneck for long context inference: at 256K tokens on a 70B model in FP16, the cache alone exceeds 30 GB. Compressing it without quality loss is the third acceleration axis.

TurboQuant (Amir Zandieh + Vahab Mirrokni, Google Research — ICLR 2026, arXiv:2504.19874) is Google's theoretically-grounded answer. The method composes two algorithms:

  • PolarQuant (AISTATS 2026, arXiv:2502.02617) — converts cache vectors to polar coordinates, eliminating the per-block normalization constant overhead that costs 1-2 extra bits per number in classical quantization
  • QJL — Quantized Johnson-Lindenstrauss (AAAI 2025, arXiv:2406.03482) — uses 1 bit of sign-only residual to remove quantization bias

Numbers (per Google blog, evaluated on Gemma + Mistral). 3 bits per element → 6× memory reduction, zero accuracy loss on LongBench / NIAH / RULER / ZeroSCROLLS / L-Eval. On H100, 4-bit TurboQuant achieves 8× attention-logits speedup vs unquantized 32-bit keys (because compute is freed by reduced memory bandwidth). Targets the Gemini KV cache bottleneck explicitly — the same machinery now also feeds Google's vector-search infrastructure at scale.

Community continuation. The asymmetric K vs V insight at the heart of TurboQuant — K cache is effectively low-rank once RoPE is undone, V is flat-spectrum — has been picked up in 2026 by two independent open-source enhancements: TurboQuant+ (Tom Turney, May 2026, PR on antirez/ds4) and Shard (Krish Garg + Kirrithan Sathananthan, May 2026, 10× compression on Llama-3.1-8B). Earlier work in the same lineage: KIVI 2024, AsymKV (Oct 2024), KVTC (Nov 2025), DeepSeek's MLA (May 2024, the same insight applied at training time rather than post-hoc).

Composes orthogonally with MTP and EAGLE. KV quant doesn't compete with spec decoding — it multiplies. A model with MTP-3 active + TurboQuant 3-bit KV cache: spec decoding saves forward passes, KV quant saves the memory per cached token, batch sizes go up because the cache occupies less, throughput goes up further.

Sources. TurboQuant paper arXiv:2504.19874 (ICLR 2026) · Google Research blog TurboQuant: Redefining AI efficiency with extreme compression (March 2026) · PolarQuant arXiv:2502.02617 · QJL arXiv:2406.03482. Community: TurboQuant+ (Tom Turney), Shard (Krish Garg) — both May 2026.

Licenses

  • Gemma 4 under Apache 2.0 is a turning point. Google moved from a restrictive custom license to standard open-source for the first time.

  • Llama 4 excludes the EU for multimodal models. But text-only Llama (3.3 70B, 3.2 1B/3B) is EU-exploitable — the exclusion only applies to multimodal.

  • "Open-weight" is more nuanced than "open-source". Llama is technically open-weight but with geographic restrictions on multimodal. Always check the fine print.


Benchmarks reference

What each benchmark measures, how many questions it has, and where to find more.

Reasoning & Knowledge

  • GPQA Diamond (198 questions) — Graduate-level questions in physics, chemistry, biology. Designed to be unsolvable by Google search. Experts reach 65%, non-experts 34%. The most discriminating reasoning benchmark.

  • MMLU-Pro (12K+ questions) — Hardened version of MMLU: 10 choices instead of 4, requires chain-of-thought reasoning. 14 domains. Drops accuracy 16-33% vs MMLU. Published at NeurIPS 2024.

Math

  • AIME (15 problems/year) — American Invitational Mathematics Examination. Competition-level math requiring creativity and multi-step reasoning. Each year's edition is harder. Only compare within the same version (2024/2025/2026).

  • MATH-500 (500 problems) — Diverse math problems (algebra, geometry, combinatorics, number theory). Good general math evaluation but easier to saturate than AIME.

Code

  • SWE-bench Verified (500 issues) — Real bugs from GitHub repos (Django, Flask, scikit-learn). The model must understand the codebase, find the bug, and produce a working patch. Human-validated by OpenAI. Paper

  • Codeforces (ELO system) — Algorithmic competition performance, scored like chess ELO. Measures pure algorithmic skill, not real-world coding. Different skill from SWE-bench.

  • LiveCodeBench (rotating, 700+) — Fresh competitive programming problems collected after model training cutoffs. Eliminates data contamination. Problems from LeetCode, AtCoder, Codeforces. GitHub

Long context

  • RULER (parametric) — Sophisticated "needle in a haystack" with multiple needles, multi-hop tracing, and aggregation. Tests at different lengths (4K to 1M). By NVIDIA. Many models claiming 1M context fail above 32K. GitHub

Agents & Tools

  • BFCL (2K+) — Berkeley Function Calling Leaderboard. Tests function/tool calling accuracy: correct names, parameters, types. V4 adds web search and memory. By UC Berkeley. GitHub

Theorem proving

  • miniF2F (488 problems) — Formal Olympiad-level math problems in Lean 4 (also Isabelle, HOL Light). Covers AMC, AIME, IMO, and university math. Proofs are compiler-verified: either correct or rejected. Zero hallucination possible. GitHub

GUI

  • ScreenSpot (1.2K+ instructions) — GUI element grounding across desktop, mobile, and web. Tests if the model can locate the right UI element from a natural language instruction. GitHub

Licenses

LicenseModelsCommercialEUPatent grantOSI
Apache 2.0Gemma 4 (incl. 12B Unified), DiffusionGemma, Qwen 3/3.5/3.8-27B, Muse-Glimmer, Kolibri-1, Raon-1, AliceAI-Foundation, Laguna M.1, North-Mini-Code, Hy3, Inkling-Small, MiniCPM5, WebWorld, GPT-OSS, Ministral, Step-3.5-Flash, NousCoder, OmniCoder, Mellum 2, Nomos 1, URM, ZAYA1, HRM-Text, Bonsai, DeepSeek-OCR-2, Lance, MiniCPM-V, Hermes 4.3-36B, Pleias (all variants), BaguettotronYesYesYesYes
MITGLM-4.5-Air, GLM-4.7-Flash, GLM-5.3-Flash, Ling-3.0, MiMo-V2.6, Motif-3, GigaChat 3.5, DeepSeek R1-Distill, DeepSeek-V4-Flash, DeepSeek-OCR (v1), Ling-2.6-flash, Kimi-Linear, MiniMax M2.7 (verify on HF), Phi-4YesYesNo (implicit)Yes
🔴 Modified MIT (revenue/MAU caps)Mistral Medium 3.5 (revenue cap), historically Kimi K2.5 (100M MAU), MiniMax M2.5ConditionalConditional--No
Nemotron OMLNemotron 3 Nano/Super, Nemotron 3 Nano OmniYesYesYesNo
OpenMDW-1.1Nemotron 3.5 (Lightning, Super-VL, Bolt), Laguna XS 2.1 / S 2.1YesYes--No
🔴 Custom (read before use)Solar Open 2 (Upstage Solar License: derivatives named "Solar…", "Built with Solar"), MiniMax-M3.1-Flash (minimax-community), Qwen3.8-Flash-Next (qwen-community-1.0)CheckCheck--No
Jamba OMLJamba 1.6YesYes--No
Llama CommunityLlama 3.3 70B, Llama 3.2 1B/3B (text-only), Hermes 4-70BYesYes (text-only)--No
LFM Open v1.0LFM2, LFM2.5, LFM2.5-VL, LFM3Yes (< $10M)Yes--No
Qwen LicenseQwen2.5-MathYesYes--No

How to choose

ConstraintRecommendation
Smartphone / edge (< 4 GB)MiniCPM5-2B, LFM3-1.7B, SmolLM3-3B, SmolLM2-135M/360M/1.7B, Gemma 4 E2B, Phi-4-mini, Ministral 3B, LFM2.5-1.2B, Llama 3.2 1B/3B
Laptop 16 GBGPT-OSS-20B, Ministral 14B, Gemma 4 26B-A4B
Desktop 24 GBQwen3.8-27B, Muse-Glimmer-30B (4-bit), Nemotron 3.5 Bolt 74B-A9B ("24GB" 2-bit build), Raon-1-mini, Gemma 4 31B, DeepSeek R1-Distill-32B, Devstral Small 2, GLM-4.7-Flash Q4 (agent coding on RTX 4090)
Desktop 48+ GB (dense 70B)Llama 3.3 70B (MMLU 86.0, EU OK), InternVL3-78B (vision), Kolibri-1 78B-A3.5B (Q4 ≈ 43 GB, MoE)
Server single-GPU (80 GB)GPT-OSS-120B, Nemotron 3.5 Super-VL (Q4 ≈ 68 GB), Laguna S 2.1
Server multi-GPUStep-3.5-Flash, Nemotron 3 Super, Qwen3.5-122B, Ling-2.6-flash 104B
Workstation 256 GB (extended tier)DeepSeek-V4-Flash 284B/A13B (native 1M ctx, FP4+FP8), Inkling-Small 276B-A12B (SWE-bench V 80.2), GLM-5.3-Flash, Motif-3, MiMo-V2.6-Flash (omnimodal)
Long context (> 256K)Nemotron 3 Nano (1M, RULER 86.3%), DeepSeek-V4-Flash (1M native)
Token-efficient agent loopsLing-2.6-flash (15M tokens on full AA suite)
On-device + thinking modeZAYA1-8B (760M active, 8.4B total)
Multimodal any-to-anyNemotron 3 Nano Omni 30B-A3B (text+audio+image+video)
OCR / long-doc compressionDeepSeek-OCR-2 (Apache 2.0, Optical Compression)
Linear-attention research baselineKimi-Linear 48B/A3B (1M ctx, KDA + MLA hybrid)
Extreme quantization (mobile)Bonsai-8B (1-bit, 1.15 GB)
Vision on edge (< 1 GB)LFM2.5-VL-450M
RL self-play research (Lean)DeepSeek-Prover-V2-7B + SGS (Stanford)
Built-in MTP (zero setup)Step-3.5-Flash (MTP-3), MiniMax M2.7 (K=3), Nemotron 3 Super, DeepSeek-V4-Flash; Gemma 4 with post-hoc drafter; Unsloth GGUF for Qwen3.6
Max speedup spec decoding (above 70B, no native MTP)EAGLE-3 external drafter (3.6× on Llama 3.1 70B), available for Llama family
KV cache memory bottleneck (long context)Google TurboQuant 3-bit (6× reduction, lossless, ICLR 2026); community: TurboQuant+ / Shard
Mobile MLLM (< 1.5 GB)MiniCPM-V 4.6 (1.2B, iOS/Android/HarmonyOS)
Encoder-free multimodalGemma 4 12B Unified (text+image+audio, no separate encoders)
Unified gen+understanding (image/video)Lance 3B (ByteDance)
IDE-resident coding (active-param efficient)Mellum 2 12B-A2.5B (JetBrains, LCB v6 69.9)
Web agent training / inference-time lookaheadWebWorld 8B/14B/32B (Qwen, world model — pair with UI-TARS)
Recurrent-architecture researchHRM-Text-1B (Sapient, dual-timescale H/L)
MathNemotron Nano 9B v2 (/think mode), GPT-OSS-120B
Code (real bugs)Muse-Glimmer-30B (SWE-bench V 76.0), Laguna M.1 / XS 2.1, Step-3.5-Flash, Devstral Small 2
Code (competition)GPT-OSS-120B (Codeforces 2622)
Multilingual (100+ langs)Qwen 3.5 (201), Qwen 3 (119)
Theorem proving (Lean 4)BFS-Prover-V2-32B (95% miniF2F)
Theorem proving (natural language)Nomos 1 (Putnam 2025: 87/120 with harness)
RAG with literal citationsPleias-RAG-1B (native citation, 1B)
Copyright-safe training dataPleias (100% public domain / CC)
German / EU-sovereignKolibri-1 (Aleph Alpha, from scratch, DE/EN tokenizer)
Diffusion LLM (parallel decoding)DiffusionGemma 26B-A4B (> 1100 tok/s H100 FP8)
Agent coding (laptop)OmniCoder-9B (Terminal-Bench 23.6%), GLM-4.7-Flash Q4
Competitive coding (LCB)NousCoder-14B (LCB v6 67.87%)
Uncensored generalistHermes 4-70B (SOTA RefusalBench)
GUI automationUI-TARS-1.5-7B (94.2% ScreenSpot)
ThroughputStep-3.5-Flash (350 tok/s)

Rejected models

ModelLicenseReason
Llama 4 (Maverick, Scout)Llama Community LicenseEU exclusion (multimodal)
Llama 3.2 Vision 11B/90BLlama Community LicenseEU exclusion (multimodal)
Llama-Nemotron-Super-49BLlama 3.3 LicenseInherits EU exclusion (multimodal base)
Qwen 3.6 PlusProprietaryClosed-source, API-only
CodestralNon-commercialResearch only
Falcon 3AmbiguousPotential 10% royalty
Kimi K2.5🔴 Modified MIT (100M MAU)Listed-with-warning candidate; left out pending updated fiche — see warning callout in Generalists for the principle
DeepSeek V3/R1 full (671B)MITQ4 ≈ 370 GB, beyond 256 GB extended cap
DeepSeek-V4-Pro (1.6T/49B)MITQ4 ≈ 800 GB, datacenter only
DeepSeek-V4.1-Flash (552B backbone + 196B Engram, 8B/16B active)MITQ4 ≈ 410 GB — the Flash line left the 256 GB tier (V4-Flash stays listed)
Kimi-K3 (2.78T)Custom (kimi-k3)Q4 ≈ 1.5 TB
GLM-5.2 / GLM-5.3 (753B)MIT / custom (glm-5.3)Q4 ≈ 415 GB — GLM-5.3-Flash is listed in the extended tier
Mistral Large 4 (1T-A52B)CustomQ4 ≈ 550 GB
Muse Spark Open (816B-A42B)Custom (muse-model-license)Q4 ≈ 450 GB — Muse-Glimmer-30B (Apache 2.0, distilled from it) is listed
Inkling (975B)Apache 2.0Q4 ≈ 525 GB — Inkling-Small is listed in the extended tier
Qwen3.8-2.4T-A95BCustom (qwen3.8-max)Q4 ≈ 1.35 TB
Nemotron 3 Ultra (550B-A55B)OpenMDW-1.1Q4 ≈ 310 GB
MiMo-V2.6-Pro (1.02T), Ling-3.1-flash (572B), Hy4-preview (780B), Beam-1.0 (501B-A23B, Reflection), K-EXAONE-2.0 (750B-A37B), LongCat-2.0 (1.78T), Step-5-Preview (604B)Mostly MIT / Apache 2.0All beyond 256 GB Q4
MiniCPM-V 4.7, Mage-VL, Unlimited-OCR / OCR-2, Intern-S2-397BApache 2.0 / MITUndeclared derivatives of another lab's model (Qwen3.5/3.6, Qwen3-4B, DeepSeek-OCR, Qwen3.5-397B layout) — this list keeps root models only
Qwen 3 235B / Qwen 3.5 397BApache 2.0235B Q4 ≈ 130 GB and 397B Q4 ≈ 218 GB technically fit extended tier — left out as MoE generalist territory is already covered by Ling-2.6-flash 104B and DeepSeek-V4-Flash with better efficiency

Tracking new releases

tools/hf_roots.py (Python ≥ 3.11, stdlib only) scans Hugging Face for root models — published by the lab that trained them — and keeps a running database of them:

  • Followed organisations, criteria and set-aside models live in config.toml (edited by hand). Every model released by a followed organisation since the date floor is fetched.
  • Root detection uses the base_model:{finetune,quantized,adapter,merge} tags. Third-party fine-tunes, quantizations, merges and known repackagers (unsloth, bartowski, mlx-community…) are dropped. Same-lab variants (-Instruct, -Base, -FP8, drafters, date suffixes) are folded into one family; models a lab derived from its own model under a new name are listed apart as lab derivatives.
  • Undeclared fine-tunes are flagged when a repo declares no base but has exactly the parameter count of an older root model from another lab.
  • Architecture from config.json for every family shown in ROOTS.md: layers, attention layout (full / sliding-window / linear / Mamba / conv counts, MLA), Q/KV heads, experts (routed × active + shared), context and RoPE scaling, MTP layers, vocabulary, dtype / quantization, vision or audio towers — and a KV-cache-per-token estimate (bf16) counting only the layers whose cache grows with the context (an upper bound, marked ≤, when the config does not declare its layout). Gated repos (access on request) cannot be read without an HF_TOKEN.
  • Persistence: data/models.json (one family per line) keeps first_seen / first_reported / gone across runs (likes are per-run values, kept out of it, so its git diff shows only real changes), so ROOTS.md opens with the families that appeared since the previous run, then the candidates per Q4 tier and license class, newest first, then what this README already lists, plus suggested organisations to follow (found by a global scan sorted by likes).
tools/hf_roots.py        # updates data/models.json and ROOTS.md (~500 API calls unauthenticated, cached 24 h)

ROOTS.md is a candidate pool, not the list: a model enters this README only after a read of its card.

Triage app. tools/triage.py serves a small local web app (http://127.0.0.1:8765, stdlib only) to go through the candidates one by one — name, publisher, domain, input modalities (text / image / video / audio), size and Q4 estimate, license, lineage warnings, the first paragraphs of the model card and the scores found in it — and keep or drop each one, with an optional reason. Decisions go to data/triage.json and are applied by the next hf_roots.py run (keep = selected for this README, or a derivative kept as an exception; drop = set aside). A Benchmarks tab compares every model (candidates and README entries) on the benchmarks you pick, sortable by any column. Scores are extracted automatically from the tables of the model cards (the column or row whose header matches the model name, names normalised: GPQA-Diamond → GPQA Diamond); each value shows the card row and column it came from, and settings (thinking mode, tools, harness, n-shot) differ between cards — check before quoting.

tools/hf_roots.py && tools/triage.py    # scan, then open the triage app

Contributing

Found an error? Missing a model? Open an issue or submit a PR.

Sources: HuggingFace, Papers With Code, official model repos and papers.


License

This list is licensed under CC-BY 4.0.

关于 About

Curated list of open-weight AI models with commercially exploitable licenses, verified benchmarks, and no EU restrictions.
curated-listgemmagpt-ossiallmmistralmodelsmoeopen-weight-modelsopensourceqwen

语言 Languages

Python65.8%
HTML34.2%

提交活跃度 Commit Activity

代码提交热力图
过去 52 周的开发活跃度
24
Total Commits
峰值: 14次/周
Less
More