llm-d - Kubernetes-Native Distributed LLM Inference with vLLM

A very common reason is a wrong site baseUrl configuration.\n

Current configured baseUrl = / (default value)\n

We suggest trying baseUrl = \n\n',document.body.prepend(n);var e=document.getElementById("__docusaurus-base-url-issue-banner-suggestion-container"),s=window.location.pathname,o="/"===s.substr(-1)?s:s+"/";e.innerHTML=o}document.addEventListener("DOMContentLoaded",function(){void 0===window.docusaurus&&insertBanner()})

llm-d: a high-performance and scalable distributed LLM inference framework

Documentation Blog Community Videos

llm-d is a well-lit path for anyone to serve at scale, with the fastest time-to-value and competitive performance per dollar, for most models across a diverse and comprehensive set of hardware accelerators.