Furiosa RNGD single card: about-10B model 2000-3000 tok/s aggregate (Hot Chips 2024 range)
2500 tok/s aggregate, midpoint of a 2000-3000 vendor range
Fields
id
benchmarks/furiosa-rngd-10b-tps-aggregate-hotchips-2024Metric
tps_aggregate
Value
2,500
Unit
tok/s aggregate, midpoint of a 2000-3000 vendor range
Model
about-10B-parameter LLM (checkpoint not named; Llama 3.1 named separately as runnable on one card)
Methodology
VENDOR RANGE, not a point measurement. Furiosa Hot Chips 2024 press release (2024-08-26): a single RNGD PCIe card delivers 2,000 to 3,000 tokens per second throughput for models around 10 billion parameters, depending on context length. Stored value is the midpoint. Engine version, batch size, quantization, and prompt and output lengths are not stated. Recorded as aggregate throughput: the sentence says throughput, and 2000-3000 tok/s on a 10B-class model is not a batch-1 decode rate. Early silicon, before the 2025 furiosa-llm stack.
Measured by
vendor
Reproducible
no
notes
ENGINE_ID NOTE: pinned to vllm, the closest existing engine. Later Furiosa-LLM is described as vLLM-compatible; this 2024 press release does not name an engine. Do not read 2500 as a measured point. Draft because model, batch, and context are unspecified. Same page quotes TDP 150 W; the product page read the same day says 180 W.
Notes
ENGINE_ID NOTE: pinned to vllm, the closest existing engine. Later Furiosa-LLM is described as vLLM-compatible; this 2024 press release does not name an engine. Do not read 2500 as a measured point. Draft because model, batch, and context are unspecified. Same page quotes TDP 150 W; the product page read the same day says 180 W.