Qwen docs: Qwen3-1.7B BF16 on 1x H20 SGLang, input 30720, 2439.03 tok/s e2e
2439.03 tok/s (batch-1 e2e, in=30720+2048gen)
Fields
id
benchmarks/qwen-bench-h20-sglang-qwen3-1p7b-bf16-in30720-tpsMetric
tps_aggregate
Value
2,439.03
Unit
tok/s (batch-1 e2e, in=30720+2048gen)
Model
Qwen3-1.7B
Methodology
VENDOR-MEASURED (Qwen team, qwen.readthedocs.io Speed Benchmark). SGLang 0.4.6.post1 + sgl-kernel 0.1.0, PyTorch 2.6.0+cu124, vLLM 0.7.2 for AWQ-marlin. NVIDIA H20 96GB, context_length=140000, enable_mixed_chunk, mem_fraction_static=0.85, skip_tokenizer_init. Batch size 1, minimum GPU count. Speed=(prompt+generated)/e2e time = end-to-end per-request throughput including prefill — NOT pure decode. Generate 2048 tokens. Config: 1x H20.
Measured by
vendor
Reproducible
yes
notes
Metric is end-to-end (prompt+gen)/time on a single request — equivalent to ~decode rate only as input_len->0; at larger input lengths it blends prefill+decode. Reads LOWER than a pure tg number. Vendor page; H20 is the China-export Hopper (96GB HBM3, ~4.0 TB/s, 148 TFLOPS BF16).
Notes
Metric is end-to-end (prompt+gen)/time on a single request — equivalent to ~decode rate only as input_len->0; at larger input lengths it blends prefill+decode. Reads LOWER than a pure tg number. Vendor page; H20 is the China-export Hopper (96GB HBM3, ~4.0 TB/s, 148 TFLOPS BF16).