nace-ai/kapitsa - 观星指南 Stargazers.cn

Kapitsa: a blazing fast Apple Silicon optimized inference engine built for Qwen and Nemotron. Writing speed of Qwen3.6 35B-A3B on a MacBook Pro (2024), M4 Pro, 48 GB, in tokens per second, at 4-bit: Kapitsa 154, oMLX with DFlash2 127, Ollama 89, LM Studio 86, MTPLX 80, llama.cpp with MTP 69.

Install

Requirements: a Mac with Apple silicon (M1 or newer) and macOS 14 Sonoma or later.

brew install nace-ai/tap/kapitsa
# or: curl -fsSL https://raw.githubusercontent.com/nace-ai/kapitsa/main/install.sh | bash

kapitsa opencode --model 35b    # also: kapitsa claude, kapitsa codex, kapitsa pi

_{On a 16-24 GB Mac use --model 9b, or 4b for the quickest replies. kapitsa serve starts the chat page and the
OpenAI and Anthropic APIs at http://127.0.0.1:8000. Kapitsa updates itself. Usage guide}

Benchmarks

Decode speed of a 512-token coding answer, in tokens per second, for Kapitsa and eight other engines on a MacBook Pro M4 Pro 48 GB, an M5 Max 128 GB and an M1 Pro 16 GB, each engine in its fastest setup; the numbers are in the tables below

_{Writing speed of a coding answer, each engine in its fastest setup. 🟩 fastest per model · — not measured ·
details}

MacBook Pro (16-inch, 2024) · M4 Pro · 48 GB

Qwen

Model	Kapitsa	_oMLX	_mlx‑serve	_Ollama	_MTPLX	_LM Studio	_llama.cpp	_MLX	_Rapid‑MLX
Qwen3.5 4B _4-bit	🟩 179 tok/s	102	137	109	104	81	70	88	98
Qwen3.8 27B _4-bit	🟩 63 tok/s	45	54	32	36	15	18	28	—
Qwen3.6 35B‑A3B _4-bit	🟩 154 tok/s	127	124	89	80	86	69	86	—

Nemotron

Model	Kapitsa	_oMLX	_llama.cpp	_MLX
Nemotron Dory1.0 4B _NVFP4	🟩 64 tok/s	— _{no support}	6	48
Nemotron 3.5 Lightning 30B‑A3B _NVFP4	🟩 106 tok/s	—	—	65
Nemotron 3.5 Bolt 74B‑A9B _{2/4/8-bit mix}	🟩 64 tok/s	35	— _{no Metal support}	32

MacBook Pro (16-inch, 2026) · M5 Max · 128 GB

Qwen

Model	Kapitsa	_oMLX	_mlx‑serve	_Ollama	_MTPLX	_LM Studio	_llama.cpp	_MLX	_Rapid‑MLX
Qwen3.5 4B _4-bit	🟩 332 tok/s	192	271	213	175	154	160	148	187
Qwen3.8 27B _4-bit	102 tok/s	93	🟩 130	68	69	32	51	54	61
Qwen3.6 35B‑A3B _4-bit	🟩 281 tok/s	232	264	166	123	127	171	132	98
Qwen3.8 Flash‑Next 180B‑A7B _4-bit	105 tok/s	107	🟩 108	60	88	— _{won't load}	96	77	— _{won't load}

Nemotron

Model	Kapitsa	_oMLX	_mlx‑serve	_Ollama	_MTPLX	_LM Studio	_llama.cpp	_MLX
Nemotron Dory1.0 4B _NVFP4	🟩 117 tok/s	— _{no support}	— _{no support}	— _{no support}	— _{no support}	— _{no support}	9	76
Nemotron 3.5 Lightning 30B‑A3B _NVFP4	🟩 239 tok/s	190	190	153	131	110	159	139
Nemotron 3.5 Bolt 74B‑A9B _{2/4/8-bit mix}	🟩 120 tok/s	69	— _{no support}	— _{no support}	— _{no support}	— _{no support}	— _{no Metal support}	69
Nemotron 3.5 Super‑VL 120B‑A12B _NVFP4	🟩 85 tok/s	52	— _{no support}	64	44	61	66	46

MacBook Pro (14-inch, 2021) · M1 Pro · 16 GB

Qwen

Model	Kapitsa	_oMLX	_mlx‑serve	_Ollama	_MTPLX	_LM Studio	_llama.cpp	_MLX	_Rapid‑MLX
Qwen3.5 4B _4-bit	🟩 85 tok/s	55	62	55	42	47	38	46	52

Nemotron

Model	Kapitsa	_oMLX	_llama.cpp	_MLX
Nemotron Dory1.0 4B _NVFP4	🟩 40 tok/s	— _{no support}	4	30

MacBook Pro (16-inch, 2026) · M5 Pro · 48 GB

Qwen

Model	Kapitsa	_oMLX	_mlx‑serve	_Ollama	_MTPLX	_LM Studio	_llama.cpp	_MLX	_Rapid‑MLX
Qwen3.5 4B _4-bit	🟩 179 tok/s	113	163	134	113	89	93	101	112
Qwen3.8 27B _4-bit	61 tok/s	51	🟩 71	40	41	17	30	32	35
Qwen3.6 35B‑A3B _4-bit	🟩 167 tok/s	144	156	102	89	96	107	98	76

Nemotron

Model	Kapitsa	_oMLX	_mlx‑serve	_Ollama	_MTPLX	_LM Studio	_llama.cpp	_MLX
Nemotron Dory1.0 4B _NVFP4	🟩 70 tok/s	— _{no support}	— _{no support}	— _{no support}	— _{no support}	— _{no support}	13	53
Nemotron 3.5 Lightning 30B‑A3B _NVFP4	🟩 136 tok/s	124	115	92	101	72	101	94
Nemotron 3.5 Bolt 74B‑A9B _{2/4/8-bit mix}	🟩 79 tok/s	39	— _{no support}	— _{no support}	— _{no support}	— _{no support}	— _{no Metal support}	41

_{Measure your Mac: kapitsa bench --submit adds it to the community results.}

Models

Model	Download	Speculation
Qwen3.5-4B · `4b` _dense	4-bit · 4.6 GB _{also 6-bit 5.6 · 8-bit 6.7 · bf16 10.6 GB}	DFlash / MTP
Qwen3.5-9B · `9b` _dense	4-bit · 9.0 GB	DFlash / MTP
Qwen3.8-27B · `27b` _dense	4-bit · 20.2 GB _{also 5-bit 23.5 · 6-bit 26.9 · 8-bit 33.6 · bf16 58.8 GB}	DFlash2 / MTP
Qwen3.6-35B-A3B · `35b` _{MoE, 3B active}	4-bit · 22.1 GB _{also 5-bit 26.4 · 6-bit 30.7 · 8-bit 39.4 · bf16 71.9 GB}	DFlash2 / MTP
Qwen3.8-Flash-Next 180B-A7B · `flash-next` _{MoE, 7B active; 128 GB Macs}	4-bit · 105.9 GB _{4/6/8-bit MLX; its 32 GB n-gram table stays on the SSD}	MTP
Nemotron Dory1.0 4B · `dory` _{looped dense, 1-4 loops per token; text only}	NVIDIA NVFP4 · 3.2 GB _{also bf16 8.0 GB}	—
Nemotron 3.5 Lightning 30B-A3B · `lightning` _{hybrid Mamba-2 MoE, 3B active; text only}	NVIDIA NVFP4 · 22.9 GB _{4-bit NVFP4 experts, FP8, BF16}	DSpark
Nemotron 3.5 Bolt 74B-A9B · `bolt` _{hybrid Mamba-2 MoE, 9B active}	NVIDIA 24 GB · 25.1 GB _{2-bit experts, NVFP4, 8-bit}	LiLiCorr
Nemotron 3.5 Super-VL 120B-A12B · `super` _{hybrid Mamba-2 MoE, 12B active; 96 GB+ Macs}	NVIDIA NVFP4 · 76.4 GB _{4-bit NVFP4 experts, FP8, BF16}	LiLiCorr

_{Other precisions: --model 35b:6bit is five times closer to bf16 than the 4-bit default (KL). All but Lightning and Dory read images. Memory: 16 GB for the 4B models, 32 GB
for the 27B, 35B, Lightning and Bolt, 96 GB for Super-VL, 128 GB for Flash-Next. Details}

How it works

How Kapitsa works. It guesses ahead, then checks in one go: a small draft model guesses the next few words, and the big model checks them all in one pass and keeps the ones it agrees with, so each pass writes several words. GPU: hand-written Metal code made for these models. Neural Engine: on the 4B, 27B, Bolt and Super-VL it reads part of each prompt beside the GPU. Memory: each conversation keeps what it has read, so a follow-up only reads the new message; up to 4 run at once.

Install

Benchmarks

MacBook Pro (16-inch, 2024) · M4 Pro · 48 GB

Qwen

Nemotron

MacBook Pro (16-inch, 2026) · M5 Max · 128 GB

Qwen

Nemotron

MacBook Pro (14-inch, 2021) · M1 Pro · 16 GB

Qwen

Nemotron

MacBook Pro (16-inch, 2026) · M5 Pro · 48 GB

Qwen

Nemotron

Models

How it works

关于 About

语言 Languages

提交活跃度 Commit Activity

核心贡献者 Contributors