vLLM on Windows in 2026: what officially works, what doesn't, and what to do once it's serving

Path 1: WSL2 (still the team's recommendation)

WSL2 is a real Linux kernel running inside Windows, with GPU passthrough to your NVIDIA card via the modern WSL CUDA driver. The vLLM team's stance, expressed across multiple GitHub issues over 2024 and 2025, is that maintaining a separate Windows build with full CUDA, Triton, and Flash-Attention kernels has high engineering cost relative to the size of the Windows-first inference audience, most of whom can use WSL2 instead. So WSL2 has remained the canonical Windows path.

What you give up: somewhere between 20 and 40 percent of native throughput on a GPU-bound workload, depending on model size, batch size, and how chatty the host-to-guest filesystem traffic is. Most of that is GPU passthrough overhead and CPU-side scheduling under Hyper-V. For a single 7B model on a single 5090 you may not notice. For batched serving on a multi-GPU box, the tax shows up.

What you gain: zero patches to track, identical behavior to a Linux server, and the entire vLLM extras ecosystem (Flash-Attention 3, FP8 quantization, Speculative Decoding) without anyone having to port kernels.

Path 2: Docker Model Runner (the easiest one)

On December 11 2025, Docker shipped vLLM as a first-class backend for Docker Model Runner on Windows. The setup is shorter than any of the other paths:

# Enable Docker Model Runner (Docker Desktop 4.54+, WSL2 enabled)
docker desktop enable model-runner --no-tcp

# Install the vLLM backend with CUDA
docker model install-runner --backend vllm --gpu cuda

# Pull and run any model that Docker Model Runner supports
docker model run ai/smollm2-vllm "Hello, Windows."

Requirements per Docker's own announcement: Docker Desktop for Windows 4.54 or newer, WSL2 backend enabled, an NVIDIA GPU with compute capability 8.0 or higher (so Ampere, Ada Lovelace, Hopper, or Blackwell), and current NVIDIA drivers. AMD and Intel GPUs are served through Vulkan in Docker Model Runner generally, but the vLLM backend specifically is NVIDIA-only.

This is still technically WSL2 underneath, so it inherits the same 20 to 40 percent virtualization tax. The win is operational: no Python environment to manage, no CUDA toolkit to install by hand, no Triton wheel hunt. If you are evaluating vLLM on Windows for the first time, this is the path that costs you the least time.

Path 3: SystemPanic/vllm-windows native wheels

The community fork at github.com/SystemPanic/vllm-windows maintains pre-built Windows wheels of vLLM plus the necessary Windows-specific kernel patches. The current release, v0.20.0, shipped on April 30 2026. Three things changed in v0.20.0 that matter:

• NCCL on Windows. Up through v0.19.0 the Windows builds were single-GPU only because Gloo was the only working distributed backend on Windows and tensor/pipeline parallelism needed NCCL.
• Tensor and pipeline parallelism. First Windows release with multi-GPU serving. This is the build that makes a 70B model on a Windows workstation with two RTX cards actually plausible.
• CUDA 13 + Blackwell support. Pre-built against Python 3.12, CUDA 13, and PyTorch 2.11. The release notes specifically call out Blackwell GPU compatibility, which the upstream Linux build also added in the same window.

The install is a single pip command against the downloaded wheel that matches your Python and CUDA versions. There is no automated installer; you pick the wheel manually from the releases page. Older v0.19.0 and v0.17.0 wheels are still up if you are pinned to CUDA 12.4 or running an earlier PyTorch.

The risk you accept: the fork is one developer, not the vLLM project. New upstream features land here some days or weeks later. Security fixes are best-effort. For a Windows-only team that genuinely cannot use WSL2 (corporate AV, driver constraints, or a kiosk-style deployment), the latency is acceptable. For a team that can use WSL2, the calculus usually pushes back toward the official path.

The part every other page on this skips

Every guide we found follows the same arc: pick a path, run the install commands, curl http://localhost:8000/v1/chat/completions, print the JSON, and end. That is enough if you only wanted to confirm the GPU works. It is not enough if you wanted vLLM for a reason.

The reason people put up with running vLLM on Windows is usually one of two things. Either they have a beefy gaming or workstation GPU on a Windows box and want to use it for something more interesting than benchmarks, or they have a private corpus or sensitive workflow they cannot send to a cloud provider. Neither use case ends at a curl. Both end with a client that does real work.

That client is what Windows is missing in 2026. Native consumer desktop agents that read the OS accessibility tree, drive other apps, and treat the local LLM as the brain still don't exist on Windows the way they do on Mac. The category is small everywhere, but on Windows it is genuinely thin. Most of what you find is browser-only, screenshot-based, or developer toolkits aimed at engineers willing to script their own agent loop. Real consumer-facing desktop automation that uses the model you just stood up is rare.

A real cross-machine pattern: Windows brain, Mac hands

If you already have vLLM serving on a Windows box and you also use a Mac, the most productive thing to do with both is to treat them as separate layers. Run inference on the Windows GPU, expose the OpenAI Chat Completions endpoint on port 8000 of the Windows machine's LAN address on the LAN, and run the agent on the Mac side where the desktop automation surface is more developed.

Fazm on the Mac exposes this directly. In Settings, AI Chat, Custom API Endpoint, you paste the URL of your Windows vLLM server (usually fronted by a thin Anthropic-shaped shim like LiteLLM, because vLLM speaks OpenAI Chat Completions and the agent's bridge talks Anthropic Messages). The Swift code that picks up that field lives in Desktop/Sources/Chat/ACPBridge.swift around lines 468 to 469:

if let customEndpoint = defaults.string(forKey: "customApiEndpoint"),
   !customEndpoint.isEmpty {
  env["ANTHROPIC_BASE_URL"] = customEndpoint
}

When that env var is set, every model call the agent makes goes to your Windows vLLM endpoint instead of api.anthropic.com. The Mac side handles the accessibility-tree work (clicking around in Finder, Calendar, Mail, Sheets), and the Windows GPU is the brain. The whole thing stays on your network. That is the realistic version of "running vLLM on Windows for an agent" in 2026, because the agent half does not exist natively on Windows yet.

One last honest take

vLLM on Windows in 2026 is a solved problem at the inference layer. Three working paths, all NVIDIA-friendly, all reach the same OpenAI-shaped endpoint. If your only question is "can I get vLLM to serve a model on my Windows box," the answer is yes, pick whichever of the three paths matches your tolerance for unofficial tooling.

What is not solved on Windows is the layer above the model: a consumer-grade desktop agent that uses the model. The pages that rank for this topic do not say so, but it is the part that matters once you are past the install. The Mac side has more there. The cross-machine Windows-brain Mac-hands pattern is the realistic way to combine them today, and it costs you exactly one field in the agent's settings.