vLLM docs: speculative decoding, quantization x hardware, structured outputs, disaggregated prefilling, multimodal inputs (five feature pages read 2026-10-04)
Fields
sources/ev-vllm-docs-feature-matrix-axesNotes
Five docs pages read 2026-10-04, used to fill the feature-comparison axes for [[vllm]] which previously recorded them only loosely. SPECULATIVE DECODING (https://docs.vllm.ai/en/latest/features/speculative_decoding/): the published method list is EAGLE, LiLiCorr, MTP, Draft Model, Parallel Draft Model (PARD), MLP speculator, N-gram, Suffix decoding, Hidden State Extraction, Custom Proposer Backend (experimental), Dynamic Speculative Decoding, Adaptive Verification, Per-Request Acceptance Metrics. The docs' own framing of the tradeoff is worth keeping because it is the axis a reader actually chooses on: 'Model-based methods such as EAGLE, MTP, draft models, PARD and MLP provide the best latency reduction, while simpler methods such as n-gram and suffix decoding provide modest speedups without increasing workload during peak traffic', and a qualitative table grades EAGLE High/Medium-high, MTP High/Medium-high, Draft model High/Medium, PARD High/Medium-high, MLP Medium/Medium-high, N-gram Low-medium/Medium, Suffix Low-medium/Medium. Docs add 'Real gains depend on your model family, traffic pattern, hardware, and sampling settings' - the project declines to publish a universal speedup, which is consistent with the repo's evidence standard. Two named incompatibilities: 'Pipeline parallelism is not composable with speculative decoding as of vllm<=0.15.0' and 'Speculative decoding with draft models is not supported in vllm<=0.10.0'. QUANTIZATION (https://docs.vllm.ai/en/latest/features/quantization/): the published implementation list is AutoAWQ, BitsAndBytes, GPTQModel, Intel Neural Compressor, LLM Compressor (FP8 W8A8, INT4 W4A16, INT8 W4A8, INT8 W8A8), NVIDIA Model Optimizer, Online Quantization, AMD Quark, Quantized KV Cache, TorchAO, FP8 ViT Encoder Attention - plus a plugin system for custom methods. Documented hardware x implementation table confirms and extends the existing per-cell caveat: AWQ is absent on Volta and on AMD GPU; GPTQ is present on Volta but absent on AMD GPU. DISAGGREGATED PREFILLING (https://docs.vllm.ai/en/latest/features/disagg_prefill/): 9 connector types named (ExampleConnector, LMCacheConnectorV1, LMCacheMPConnector, NixlConnector with UCX/GDS backends, and others); states the two reasons it exists (tune TTFT and ITL independently; control tail ITL) and carries a caveat worth preserving because it is a marketing-shaped claim's correction - 'Disaggregated prefill DOES NOT improve throughput'. Also notable: the docs concede 'Disaggregated prefilling is highly related to infrastructure, so vLLM relies on third-party connectors for production-level disaggregated prefilling', which qualifies the earlier record's implication that the feature is production-ready in-tree. STRUCTURED OUTPUTS (https://docs.vllm.ai/en/latest/features/structured_outputs/): backends are xgrammar and guidance (llguidance); 'Structured outputs are supported by default in the OpenAI-Compatible Server'; selectable per-request via choice / regex / json / grammar / structural_tag; --structured-outputs-config.backend default is 'auto'. Also records the API break: guided_json, guided_regex, guided_choice, guided_grammar, guided_whitespace_pattern, structural_tag and guided_decoding_backend were REMOVED in v0.12.0 in favour of a single structured_outputs object - a real upgrade hazard for anyone on an older release. MULTIMODAL INPUTS (https://docs.vllm.ai/en/latest/features/multimodal_inputs/): image and video paths including image_pil and video_url; security guidance the record previously lacked - set --allowed-media-domains to restrict fetchable domains against SSRF and VLLM_MEDIA_URL_ALLOW_REDIRECTS=0 to block redirect bypass, 'especially important if you run vLLM in a containerized environment where the vLLM pods may have unrestricted access to internal networks'.
Cross-references
This record references no other records.