ovg-project
Github

代码库

ovg-project/kvcached
1.5k
Virtualized Elastic KV Cache for Dynamic GPU Sharing and Beyond
Python
elastic-kvcachegpu-mutiplexinggpu-sharinginference-enginekvcachekvcache-optimizationkvcachedllmllm-frameworkllm-inferencellm-servingollamaonline-offline-coserveserverlesssglangvllm