XLA

XLA (Accelerated Linear Algebra) is Google's ML compiler, originally developed for TensorFlow and now part of the OpenXLA project. It is the only production compiler that targets TPUs, making it essential for TPU-based inference. For NVIDIA GPUs, XLA is less commonly used directly; instead, JAX and PyTorch models are often compiled with Triton or hand-written CUDA kernels. XLA's strength is its integration with the TensorFlow/JAX ecosystem and its mature optimization pipeline. However, for LLM inference on NVIDIA GPUs, XLA is not the primary compilation path. The lack of autotuning is a significant disadvantage: kernel performance on modern GPUs is highly sensitive to tile sizes, and XLA's heuristics often miss optimal configurations. XLA is also not designed for inference-specific optimizations like KV cache management or continuous batching, which are implemented at the engine level (e.g., vLLM, TensorRT-LLM) rather than in the compiler.