https://kre80r.com/tags/capacity-planning/Recent content in Capacity-Planning on /sys/admin/blogHugoenSat, 10 Oct 2026 00:00:00 +0000https://kre80r.com/room-for-two/Sat, 10 Oct 2026 00:00:00 +0000https://kre80r.com/room-for-two/<p>With vLLM running and nobody connected, <code>nvidia-smi</code> shows this:</p> <div class="highlight"><pre tabindex="0" class="chroma"><code class="language-text" data-lang="text"><span class="line"><span class="cl">GPU 0: 22,112 MiB / 24,576 MiB used, 0% utilisation </span></span><span class="line"><span class="cl">GPU 1: 22,134 MiB / 24,576 MiB used, 0% utilisation </span></span></code></pre></div><p>My first thought was a memory leak, but vLLM takes that memory on purpose. My config sets <code>--gpu-memory-utilization 0.90</code>, so when vLLM starts it claims 90% of each GPU and manages that space itself. &ldquo;Memory used&rdquo; tells me nothing about load.</p>