SyFI Lab (UW) — VibeSys Builds a Qwen3.5-397B Engine on MI300A, 2.3x Tuned SGLang

Fields

id

sources/syfi-qwen35-397b-mi300a-vibesys

URL

https://syfi.cs.washington.edu/blog/2026-10-08-qwen35-mi300a/

Publisher

SyFI Lab, University of Washington

Kind

blog

Published

2026-10-08

Accessed

2026-10-15

notes

4x AMD Instinct MI300A (128GB HBM3 shared CPU+GPU each, gfx942). Model: Qwen3.5-397B-A17B MXFP4 (amd/Qwen3.5-397B-A17B-MXFP4), 397B params, ~17B active, 512 experts, 15 full-attn + 45 Gated-DeltaNet linear-attn layers. Workload: deterministic multi-turn chat, 1->192 concurrent sessions, 3-6 turns of 80-300 gen tokens each, full history resent per turn. SLO: p95 TPOT<=250ms, p95 TTFT<=10s. Goodput = tok/s of responses meeting SLO. Baseline: VibeSys-ported+tuned SGLang v0.5.18 (961 tok/s @48 sessions); bespoke agent engine 2242 tok/s @96 sessions.

Notes

4x AMD Instinct MI300A (128GB HBM3 shared CPU+GPU each, gfx942). Model: Qwen3.5-397B-A17B MXFP4 (amd/Qwen3.5-397B-A17B-MXFP4), 397B params, ~17B active, 512 experts, 15 full-attn + 45 Gated-DeltaNet linear-attn layers. Workload: deterministic multi-turn chat, 1->192 concurrent sessions, 3-6 turns of 80-300 gen tokens each, full history resent per turn. SLO: p95 TPOT<=250ms, p95 TTFT<=10s. Goodput = tok/s of responses meeting SLO. Baseline: VibeSys-ported+tuned SGLang v0.5.18 (961 tok/s @48 sessions); bespoke agent engine 2242 tok/s @96 sessions.

Cross-references

This record references no other records.