vLLM v0.19.0 from the viewpoint of a Mac tool-calling agent

vLLM v0.19.0 was tagged on April 3, 2026 with 448 commits from 197 contributors. The headline items are Gemma 4 support across the full family, Model Runner V2 maturation with piecewise CUDA graphs for pipeline parallelism, zero-bubble async scheduling that is finally compatible with speculative decoding, a new /v1/chat/completions/batch endpoint for offline workloads, CPU KV cache offloading with pluggable eviction policies, online MXFP8 quantization for MoE and dense models, and NVIDIA B300 and GB300 support. There are also new architectures (Cohere ASR, ColQwen3.5, Granite 4.0 Speech) and new tool parsers (GigaChat, Kimi-K2.5, Gemma 4).

Three features in v0.19.0 move the needle specifically for tool-calling loops. First, zero-bubble async scheduling + speculative decoding can now ship together. Before this release you had to pick one; speculative decoding pulled latency down on individual tokens but gave up the scheduling gains across concurrent requests. For an agent that round-trips through model -> tool -> model -> tool on every user turn, inter-call latency dominates. Second, the new /v1/chat/completions/batch endpoint fits agent-trace replay workloads cleanly; you can run a session against a frontier model, capture the trace, then batch-replay the prompts through a local open model to generate training data or evals without real-time latency constraints. Third, CPU KV cache offloading means a single 48GB M-series box can hold far more in-flight contexts than before, which is relevant for running an eval suite locally.

Indirectly, yes. Fazm's ACP bridge reads one environment variable to redirect the agent's brain: at Desktop/Sources/Chat/ACPBridge.swift line 380 it writes env["ANTHROPIC_BASE_URL"] = customEndpoint. Set Settings -> AI Chat -> Custom API Endpoint to the URL of an Anthropic-to-OpenAI translator (claude-code-router, LiteLLM proxy, or similar) that forwards Messages API calls to vLLM's /v1/chat/completions. Fazm continues to call whatever model name is selected (the default at acp-bridge/src/index.ts line 1126 is claude-sonnet-4-6), the translator maps the name to a vLLM-served model, and the rest of the loop is unchanged. Tool calling survives because v0.19.0 ships a Gemma 4 tool parser and the generic Qwen3 tool parser, which most translator layers can map to the Anthropic tool_use shape.

Online MXFP8 quantization on a model that is actively used for tool calling is a trap. Quantization degrades structured-output fidelity unevenly; the arguments field of tool calls is exactly where hallucinated or malformed JSON shows up first. The v0.19.0 implementation is solid for chat completions, but you want to eval your specific agent against the quantized version before switching it in as the tool-calling driver. Benchmarks that report accuracy on MMLU or HumanEval tell you very little about what happens at the fourteenth tool call in a session.

Yes, favorably in most cases. Piecewise CUDA graphs mean the runner can capture distinct execution paths for different request shapes without monolithic recompilation. For a tool-calling agent, turns have heterogeneous shapes (short acknowledgments, long accessibility-tree reads, long tool-result summaries). Under MRV1 the runner would either over-pad or re-enter eager mode; under MRV2 each shape gets its own captured graph and you stop paying the transition cost. The caveat is that MRV2 is maturing, not the default in every config, and some pipeline parallelism setups need config adjustments when you opt in.

Not yet for the primary loop. The default at acp-bridge/src/index.ts line 1126 is claude-sonnet-4-6 because that model has the tool-call throughput and reliability to survive twenty-plus sequential tool invocations per user turn without the loop collapsing. Gemma 4 26B MoE and Qwen3.5 in v0.19.0 get closer, but sustained structured tool-calling over long accessibility-tree payloads still favors hosted frontier models. The interesting role for a local v0.19.0 install today is as the replay engine for eval and the cheaper sidecar for guardrail classification, not as the main brain.

The agent itself stays on its real-time backend (frontier model for reliability). Separately, you record sessions: every turn, every tool call, every tool result, every accessibility-tree fragment the model was shown. You reshape those traces into batch-API requests and submit them to your local vLLM v0.19.0 instance. The batch endpoint lets you reprocess thousands of turns through an open model overnight, building eval corpora or fine-tuning data without blowing up your hosted-model bill. This is the workflow the release notes do not spell out but the feature is clearly shaped for.