vLLM compressed-tensors W4A8 dispatch code (compressed_tensors.py, schemes/compressed_tensors_w4a8_int.py, kernels/linear/mixed_precision/*): how a W4A8 checkpoint is routed and where it fails

Read from the vLLM main branch 2026-10-04. This is the code that decides what a W4A8 (INT4 weight + INT8 activation) checkpoint actually runs on, and it is the evidence that the Implementation x Hardware docs table's 'llm-compressor INT8 (W4A8)' row understates CPU availability. The relevant facts, quoted. (1) The scheme is selected by CompressedTensorsConfig._get_scheme_from_parts via the predicate _is_dynamic_token_w4a8_int(weight_quant, input_quant), which requires weight INT 4-bit group-or-channel, input INT 8-bit with strategy TOKEN, weight dynamic False, input dynamic True, weight symmetric. It returns CompressedTensorsW4A8Int. (2) CompressedTensorsW4A8Int.get_min_capability() returns 1 - the LOWEST possible value, and deliberately so: the class does not gate on GPU capability at all, because the kernel it wants is chosen by choose_mp_linear_kernel(). (3) Inside CompressedTensorsW4A8Int.create_weights it builds an MPLinearLayerConfig with weight_type=int4, act_type=params_dtype and calls choose_mp_linear_kernel(mp_linear_kernel_config). (4) The CUDA candidate list for that mixed-precision registry is, in order: CutlassW4A8LinearKernel, MacheteLinearKernel, MarlinLinearKernel, ConchLinearKernel, ExllamaLinearKernel, TritonW4A16LinearKernel, HummingLinearKernel. Every one of those returns can_implement()=False for a W4A8 config: CutlassW4A8LinearKernel requires act_type==float8_e4m3fn (so an INT8-activation checkpoint fails it), MacheteLinearKernel requires is_device_capability(90) and act dtypes FP16/BF16/FP8 only, MarlinLinearKernel supports weight-only quant types and no input_quant_key so it cannot apply activation quantization, ConchLinearKernel requires the conch-triton-kernels package and supports weight-only group sizes [-1,128], ExllamaLinearKernel 'only supports float16 activations', TritonW4A16LinearKernel is weight-only, HummingLinearKernel is weight-only. So on CUDA the selection loop falls through all seven and raises ValueError('Failed to find a kernel that can implement the WNA16 linear layer. Reasons: ...') - a hard load-time failure with a per-kernel reason list, not a silent fallback. That is the engine-source proof that the docs row's five NVIDIA NO cells are correct as a hardware statement, while the x86 CPU NO cell is not. (5) The CPU candidate list is, in order: Dynamic4bitLinearKernel, ZentorchWNA16LinearKernel, CPUWNA16LinearKernel. Dynamic4bitLinearKernel - described in its own module docstring as 'the KleidiAI-accelerated w4a8int quantization scheme' - can_implement() requires only is_cpu() and weight_type in [scalar_types.int4], with the Arm-specific extras (float32/bfloat16/float16 activations, torch.ops.aten._dyn_quant_matmul_4bit) applied only when current_platform.get_cpu_architecture()==CpuArchEnum.ARM. There is no x86 exclusion and no ISA gate in can_implement. (6) The second x86 CPU W4A8 path is in CPUWNA16LinearKernel.process_weights_after_loading: layer.use_w4a8 = envs.VLLM_CPU_INT4_W4A8 and act_type==torch.bfloat16 and (torch.cpu._is_amx_tile_supported() or architecture is RISCV). When set, it calls _process_gptq_weights_w4a8 (repack to AWQ blocked format via ops.convert_weight_packed_scale_zp with ops.CPUQuantAlgo.AWQ) and apply_weights dispatches to ops.int4_scaled_mm_cpu instead of ops.cpu_gemm_wna16. VLLM_CPU_INT4_W4A8 defaults to True (envs.py: lambda: bool(int(os.getenv('VLLM_CPU_INT4_W4A8', '1')))). (7) The x86 C++ op exists: csrc/cpu/sgl-kernels/gemm_int4.cpp defines at::Tensor int4_scaled_mm_cpu(x, w, w_zeros, w_scales, bias) delegating to int4_scaled_mm_cpu_with_quant, which quantizes activations itself ('act quant', 'activation quants into uint8_t', constexpr bool sym_quant_act = false) and errors with 'DA8W4: weight and input shapes mismatch' / 'int4_scaled_mm_cpu_with_quant: expect A to be bfloat16 or half.'. Its arch guards are CPU_CAPABILITY_AVX512 and CPU_CAPABILITY_RVV - x86 AVX512, not AMX and not Arm. Summary of the code-level truth: W4A8 on x86 CPU exists through two independent routes, an AMX-tile-gated bf16-activation route and a general AVX512 route, and the docs table marks x86 CPU as NO.