代码库

kvcache-ai/ktransformers

17k

A Flexible Framework for Experiencing Heterogeneous LLM Inference/Fine-tune Optimizations

Python

kvcache-ai/Mooncake

5.1k

Mooncake is the serving platform for Kimi, a leading LLM service provided by Moonshot AI.

C++

disaggregationinferencekvcachellmrdmasglangvllm