Qwen docs — Speed Benchmark: Qwen3 family on NVIDIA H20 96GB (SGLang + HF Transformers)

Vendor benchmark page. Hardware: NVIDIA H20 96GB. Two stacks: (a) SGLang 0.4.6.post1 + sgl-kernel 0.1.0 + PyTorch 2.6.0+cu124 + vLLM 0.7.2 (AWQ-marlin), context_length=140000, enable_mixed_chunk=True, mem_fraction_static=0.85, skip_tokenizer_init with input_ids; (b) HF Transformers 4.51.3 + FlashAttention 2.7.4 + PyTorch 2.6.0, GPTQModel 2.2.0 / AutoAWQ 0.2.9. Batch size 1, minimum GPU count. Speed = (prompt+generated tokens)/e2e time = end-to-end per-request throughput INCLUDING prefill, NOT pure decode. Generate 2048 tokens, input lengths 1/6144/14336/30720/63488/129024.