RK3588 RKLLM Phi3-3.8B decode (w8a8)
7.45 tok/s decode (w8a8, 128-in/64-out)
Fields
id
benchmarks/comm-rk3588-phi3-3p8b-decodeMetric
decode_tok_s
Value
7.45
Unit
tok/s decode (w8a8, 128-in/64-out)
Model
Phi3-3.8B
Methodology
Vendor RKLLM benchmark table (rknn-llm benchmark.md), RK3588 NPU, w8a8 quantisation, ISL=128/OSL=64
Measured by
vendor
Thermal state
unknown
notes
VENDOR-MEASURED. Largest model in the table is ChatGLM3-6B at ~5 tok/s; a real 7B lands ~3-4 tok/s est.
Notes
VENDOR-MEASURED. Largest model in the table is ChatGLM3-6B at ~5 tok/s; a real 7B lands ~3-4 tok/s est.