SyFI/VibeSys: 4x MI300A, tuned SGLang 0.5.18, Qwen3.5-397B-A17B MXFP4, 48 sessions, goodput
961.0 tok/s aggregate goodput (48 concurrent sessions, SLO-filtered)
Fields
id
benchmarks/syfi-mi300a-sglang-tuned-qwen35-397b-mxfp4-48sess-goodputMetric
tps_aggregate
Value
961
Unit
tok/s aggregate goodput (48 concurrent sessions, SLO-filtered)
Model
amd/Qwen3.5-397B-A17B-MXFP4
Methodology
THIRD-PARTY MEASURED (SyFI Lab, UW). VibeSys-ported+tuned SGLang v0.5.18 on 4x AMD MI300A (128GB HBM3 each, gfx942). Qwen3.5-397B-A17B MXFP4, ~17B active, 512 experts, MTP speculative decoding. Deterministic multi-turn chat ramping to 192 sessions; SLO p95 TPOT<=250ms & p95 TTFT<=10s. Peak goodput 961 tok/s at 48 sessions = AGGREGATE across sessions meeting SLO.
Measured by
third_party
Reproducible
yes
notes
Stock SGLang 0.5.18 cannot serve this model on MI300A (no gfx942 MXFP4 MoE kernel); VibeSys agents wrote the fused MoE+GEMM kernels. Baseline PyTorch engine was 12.8 tok/s.
Notes
Stock SGLang 0.5.18 cannot serve this model on MI300A (no gfx942 MXFP4 MoE kernel); VibeSys agents wrote the fused MoE+GEMM kernels. Baseline PyTorch engine was 12.8 tok/s.