4x Furiosa RNGD: EXAONE 3.5 32B, batch 1, 50 tok/s at 32K context
50 tok/s, per-request, batch 1, 4-card tensor parallel
Fields
id
benchmarks/furiosa-4xrngd-exaone35-32b-32k-b1-decodeMetric
decode_tok_s
Value
50
Unit
tok/s, per-request, batch 1, 4-card tensor parallel
Model
EXAONE 3.5 32B
Methodology
VENDOR, same run as the 4K cell. One server, four RNGD cards, tensor parallel, batch size one. EXAONE 3.5 32B at a 32K context window: 50 tokens per second. Furiosa-LLM (vLLM-compatible); build not named. Prompt and generation lengths inside the 32K window are not stated. System rate at batch 1, not per-card and not an aggregate.
Measured by
vendor
Reproducible
no
notes
Pair to the 4K record. The drop from 60 to 50 tok/s when the context window goes from 4K to 32K is the whole of what the source says about context scaling. Quantization is not stated.
Notes
Pair to the 4K record. The drop from 60 to 50 tok/s when the context window goes from 4K to 32K is the whole of what the source says about context scaling. Quantization is not stated.