Jetson AGX Thor SGLang: Qwen3.6-35B-A3B FP8 TTFT warm (Multimodalflow)
101.0 ms (time to first token, warm, per-request)
Fields
id
benchmarks/thor-sglang-qwen36-35b-fp8-ttftMetric
ttft_ms
Value
101
Unit
ms (time to first token, warm, per-request)
Model
Qwen3.6-35B-A3B-FP8
Methodology
Multimodalflow live-hardware post dated 2026-06-01 on a Jetson AGX Thor Developer Kit. CUDA 13.0, compute capability 11.0, unified memory reported as 125,771 MiB (~123 GB), kernel 6.8.12-tegra. Not the MLPerf agentic-edge submission already in this repo. Same SGLang server as the decode record. Streaming TTFT, 3 runs: cold 0.282 s, warm 0.101 s, warm 0.101 s. This record is the warm figure (101 ms). The post also averages all three runs to 0.161 s; that average mixes the cold start and is not what is stored. Prompt length for the TTFT runs is not stated. Per request.
Measured by
third_party
Reproducible
yes
notes
Warm TTFT only. Cold start (282 ms) is in the notes rather than a second record because the post does not separate model-load from first-token compute. Prompt length unknown, so this is not comparable to a fixed-prefill TTFT.
Notes
Warm TTFT only. Cold start (282 ms) is in the notes rather than a second record because the post does not separate model-load from first-token compute. Prompt length unknown, so this is not comparable to a fixed-prefill TTFT.