NVIDIA DGX-1 (8x V100 32GB) Llama-3.3-70B Q4 decode (estimated)
40.0 tok/s decode range 30-50 (est), tensor-parallel over NVLink plane
Fields
id
benchmarks/comm-dgx1-8x-v100-llama33-70b-q4-decode-estMetric
decode_tok_s
Value
40
Unit
tok/s decode range 30-50 (est), tensor-parallel over NVLink plane
Model
Llama-3.3-70B Q4-class
Methodology
ESTIMATE: 8x V100-32GB TP=8, ~900 GB/s per GPU, GPTQ/AWQ 4-bit via vLLM or TGI. Volta has no FP8/INT4 tensor cores and FA2 does not target Volta; value is the midpoint of the 30-50 t/s band quoted in research-big-memory.md.
Measured by
third_party
Thermal state
unknown
notes
NOT MEASURED on this exact config - estimate only, hence status=draft. Volta-era software stack is the real constraint.
Notes
NOT MEASURED on this exact config - estimate only, hence status=draft. Volta-era software stack is the real constraint.