Qwen docs: Qwen3-14B AWQ-INT4 on 1x H20 HF-Transformers, input 30720, 363.25 tok/s e2e
363.25 tok/s (batch-1 e2e, in=30720+2048gen)
Fields
id
benchmarks/qwen-bench-h20-transformers-qwen3-14b-awq-int4-in30720-tpsMetric
tps_aggregate
Value
363.25
Unit
tok/s (batch-1 e2e, in=30720+2048gen)
Model
Qwen3-14B
Methodology
VENDOR-MEASURED (Qwen team, qwen.readthedocs.io Speed Benchmark). HF Transformers 4.51.3 + FlashAttention 2.7.4 + PyTorch 2.6.0, GPTQModel 2.2.0 / AutoAWQ 0.2.9. NVIDIA H20 96GB, batch size 1, generate 2048 tokens. Speed=(prompt+generated)/e2e time = end-to-end throughput including prefill — NOT pure decode. Config: 1x H20.
Measured by
vendor
Reproducible
yes
notes
Metric is end-to-end (prompt+gen)/time on a single request — equivalent to ~decode rate only as input_len->0; at larger input lengths it blends prefill+decode. Reads LOWER than a pure tg number. Vendor page; H20 is the China-export Hopper (96GB HBM3, ~4.0 TB/s, 148 TFLOPS BF16). HF Transformers eager path (NOT a serving engine) — far below SGLang; this isolates the framework gap on H20.
Notes
Metric is end-to-end (prompt+gen)/time on a single request — equivalent to ~decode rate only as input_len->0; at larger input lengths it blends prefill+decode. Reads LOWER than a pure tg number. Vendor page; H20 is the China-export Hopper (96GB HBM3, ~4.0 TB/s, 148 TFLOPS BF16). HF Transformers eager path (NOT a serving engine) — far below SGLang; this isolates the framework gap on H20.