llm-d

Githubhttps://llm-d.aiFollowers 608

代码库

llm-d/llm-d-inference-sim

161

A lightweight, configurable, and real-time simulator designed to mimic the behavior of vLLM without the need for GPUs or running actual heavy models.

Go

incubating

llm-d/llm-d

3.5k

Achieve state of the art inference performance with modern accelerators on Kubernetes

Shell

aicncfdistributed-inferencegpuinferenceintelligent-routingkubernetesllmmodel-server