Gaudi Static Shape Requirement for Inference Optimization

Gaudi's compiler (SynapseAI) requires static shapes for optimal kernel generation. Unlike CUDA, which can handle dynamic shapes with some overhead, Gaudi's TPC (Tensor Processor Core) kernels are compiled for specific shapes and any shape change triggers recompilation. This is why vLLM on Gaudi uses a bucketing mechanism: requests are grouped into fixed-size buckets (e.g., batch sizes 1, 2, 4, 8, 16) and each bucket has its own compiled graph. The vLLM Gaudi plugin's bucketing mechanism is listed as a key advantage. For llama.cpp on Intel GPUs (Arc, Arc Pro), the SYCL backend also benefits from static shapes but handles them differently: the SYCL runtime JIT-compiles kernels for each shape encountered, and the SYCL_CACHE_PERSISTENT environment variable can cache these (though it is not recommended due to crash risk). The static shape requirement is a fundamental architectural difference from CUDA and is the primary reason Gaudi inference requires more upfront tuning. RESOLVED (2026-10-04 cross-reference repair): affected_by_hardware values 'gaudi-2' -> intel-gaudi2 and 'gaudi-3' -> intel-gaudi3. Each value is the architecture string of exactly one accelerator record and this repo holds one card per Gaudi generation, so the mapping is forced - note it is forced by the repo's contents, not by the source document. RESOLVED 2026-10-04 by DELETION, not by repointing, and the deletion is the correct action rather than a dodge. 'hpu' names the Habana Processing Unit - the DEVICE CLASS spanning every Gaudi generation - not any card. This field already carried BOTH real targets, intel-gaudi2 and intel-gaudi3, before this pass, so 'hpu' was a third entry adding no hardware scope that was not already claimed. It was NOT repointed to either Gaudi record because the referring context does not choose between them: all three of these records are sourced to the vLLM Gaudi plugin README and plugin-system documentation (sources xpu-vllm-gaudi-readme, xpu-vllm-gaudi-plugin-system), which are CURRENT-GENERATION-AGNOSTIC - the plugin supports Gaudi 2 and Gaudi 3 with the same hpu_* custom ops and the same HPU-graph mechanism on both. Choosing intel-gaudi3 would have been a guess dressed as a fix, and it would have added a SECOND Gaudi 3 reference to a field that already had one, implying a specificity the source does not have. The cost of deleting rather than repointing is nil in this field, because the coverage 'hpu' was standing in for - 'this applies to Gaudi hardware' - is already carried by the two sibling entries. The concept itself is NOT lost: it remains in gotchas xpu-gaudi-fork-after-driver-init and xpu-gaudi-fork-eol, whose concepts arrays carry 'hpu' as free-text scope, which is where a device-CLASS name belongs since concepts are explicitly not id-resolved. A future reader who wants the device-class grouping should look for it there, not in an id field that can only hold card ids. === SOURCE DECLARED 2026-10-10 (w10-xref-consistency). === This record's notes cite 'xpu-vllm-gaudi-plugin-system' by id, and that source file exists and was read - but it was never added to this record's sources[]. The citation is load-bearing, not incidental: the paragraph above justifies DELETING 'hpu' from affected_by_hardware on the grounds that the plugin documentation is generation-agnostic, so the source for that argument was itself undeclared. Added to sources[]. No number in this record changed.