Apple M2 Max (38c, 96GB) llamafile: Llama-3.1-8B Q4_K_M decode+prefill
46.41 tok/s avg generation (batch-1); prefill 484.06 t/s
Fields
id
benchmarks/localscore-m2-max-38c-96gb-llama31-8b-q4km-decodeMetric
decode_tok_s
Value
46.41
Unit
tok/s avg generation (batch-1); prefill 484.06 t/s
Model
llama-3-1-8b-instruct Q4_K_M
Methodology
localscore.ai accelerator/515 measured row, exported via siliconscore benchmarks.csv: llamafile, Q4_K_M, M2 Max 38-core 96 GB. Per-request batch-1.
Measured by
third_party