# LLM Serving ## 2024 ### OSDI - Taming Throughput-Latency Tradeoff in LLM Inference with Sarathi-Serve [[paper]](https://www.usenix.org/conference/osdi24/presentation/agrawal) - ServerlessLLM: Low-Latency Serverless Inference for Large Language Models [[paper]](https://www.usenix.org/conference/osdi24/presentation/fu) - InfiniGen: Efficient Generative Inference of Large Language Models with Dynamic KV Cache Management [[paper]](https://www.usenix.org/conference/osdi24/presentation/lee) - Llumnix: Dynamic Scheduling for Large Language Model Serving [[paper]](https://www.usenix.org/conference/osdi24/presentation/sun-biao) - DistServe: Disaggregating Prefill and Decoding for Goodput-optimized Large Language Model Serving [[paper]](https://www.usenix.org/conference/osdi24/presentation/zhong-yinmin) - dLoRA: Dynamically Orchestrating Requests and Adapters for LoRA LLM Serving [[paper]](https://www.usenix.org/conference/osdi24/presentation/wu-bingyang) - Parrot: Efficient Serving of LLM-based Applications with Semantic Variable [[paper]](https://www.usenix.org/conference/osdi24/technical-sessions) - USHER: Holistic Interference Avoidance for Resource Optimized ML Inference [[paper]](https://www.usenix.org/conference/osdi24/presentation/shubha) - Fairness in Serving Large Language Models [[paper]](https://www.usenix.org/conference/osdi24/presentation/sheng)