Qwen docs: Qwen3-1.7B BF16 on 1x H20 HF-Transformers, input 1, 59.83 tok/s e2e
59.83 tok/s (batch-1 e2e, in=1+2048gen)
Fields
id
benchmarks/qwen-bench-h20-transformers-qwen3-1p7b-bf16-in1-tpsMetric
tps_aggregate
Value
59.83
Unit
tok/s (batch-1 e2e, in=1+2048gen)
Model
Qwen3-1.7B
Methodology
VENDOR-MEASURED (Qwen team, qwen.readthedocs.io Speed Benchmark). HF Transformers 4.51.3 + FlashAttention 2.7.4 + PyTorch 2.6.0, GPTQModel 2.2.0 / AutoAWQ 0.2.9. NVIDIA H20 96GB, batch size 1, generate 2048 tokens. Speed=(prompt+generated)/e2e time = end-to-end throughput including prefill — NOT pure decode. Config: 1x H20.
Measured by
vendor
Reproducible
yes
notes
Metric is end-to-end (prompt+gen)/time on a single request — equivalent to ~decode rate only as input_len->0; at larger input lengths it blends prefill+decode. Reads LOWER than a pure tg number. Vendor page; H20 is the China-export Hopper (96GB HBM3, ~4.0 TB/s, 148 TFLOPS BF16). HF Transformers eager path (NOT a serving engine) — far below SGLang; this isolates the framework gap on H20.
Notes
Metric is end-to-end (prompt+gen)/time on a single request — equivalent to ~decode rate only as input_len->0; at larger input lengths it blends prefill+decode. Reads LOWER than a pure tg number. Vendor page; H20 is the China-export Hopper (96GB HBM3, ~4.0 TB/s, 148 TFLOPS BF16). HF Transformers eager path (NOT a serving engine) — far below SGLang; this isolates the framework gap on H20.