RK3588 ChatGLM3-6B token-gen (RKLLM w8a8)
4.98 tok/s w8a8 128-in/64-out
Fields
id
benchmarks/comm-rk3588-chatglm3-6b-tgMetric
decode_tok_s
Value
4.98
Unit
tok/s w8a8 128-in/64-out
Model
ChatGLM3-6B
Methodology
Rockchip rknn-llm benchmark.md table, RK3588 w8a8 quant, 128 input / 64 output
Measured by
vendor
Thermal state
unknown
notes
Largest they benchmark, already ~5 t/s. Qwen2.5-1.5B 16.7; Phi3-3.8B 7.45; Qwen2-0.5B 41.6. A real 7B lands ~3-4 t/s - the wall. (RKLLM runtime - no dedicated engine record; engine_id is a placeholder.)
Notes
Largest they benchmark, already ~5 t/s. Qwen2.5-1.5B 16.7; Phi3-3.8B 7.45; Qwen2-0.5B 41.6. A real 7B lands ~3-4 t/s - the wall. (RKLLM runtime - no dedicated engine record; engine_id is a placeholder.)