Senn Tech: vLLM prefix caching measured on self-hosted Qwen3.8-27B

Operator blog by Franz Senn, dated 2026-09-02, read 2026-10-18. They enabled --enable-prefix-caching and --enable-prompt-tokens-details on a local Qwen3.8-27B on 19 August 2026. After four days on a 4x RTX 5090 host (128 GB VRAM) the Prometheus counters read roughly 78 million prefill tokens, 5.9 million generated tokens, and a prefix cache hit rate of about 43-45 percent; reused long documents or tool schemas hit 70-90 percent cached, short ad-hoc requests near zero. The single-request measurement this record uses is the separate hermes host, one RTX 5090: a 150,000-token replay prompt with identical system instructions and tool schema dropped from 53.2 s to 2.3 s at a measured cache hit rate of 93.5 percent. The post does not name the vLLM version, the weight quant, or whether 53.2 s is TTFT or end-to-end, so confidence is 0.62 and the kv record treats 53.2 s as prefill time of a replay, not a decode rate.