Distributed LLM Inference logopeterindia.net
Knowledge Hub

Distributed LLM Inference

Distributed LLM inference on Kubernetes: vLLM makes one GPU fast, llm-d makes a fleet of GPUs efficient, with cache-aware routing, prefill/decode split and KV cache offloading.

8 topics•3 categories•Updated November 22, 2026

Spotlight: The InfoQ Podcast

The episode this page is built from. Summaries follow the episode page and transcript.

Projects Discussed

3 entries

vLLM makes one GPU fast, and llm-d makes a fleet of GPUs efficient. Open source projects covered in the episode, with one link each.