inference/research
verifiedhigh · 0.90updated 2026-10-04

KTransformers README: prefix cache, GLM-5.3-flash multimodal on consumer GPUs (read 2026-10-04)

blog

Fields

id
sources/ev-ktransformers-readme-feature-axes
Publisher
kvcache-ai
Kind
blog
Accessed
2026-10-04
notes
Read 2026-10-04, used to fill the feature-comparison axes for [[ktransformers]]. PREFIX CACHING is stated with its hardware extent, which is the interesting part: 'June 30, 2025: Support 3-layer (GPU-CPU-Disk) prefix cache reuse' - a three-tier prefix cache that spills from GPU to CPU to DISK. That is a different mechanism from every other engine in this repo, whose prefix caches are in-memory only, and it is the natural answer to 'why does this engine exist': CPU/Disk tiers exist because its weights do not fit in VRAM at all, so a KV cache that outlived GPU memory was necessary rather than nice. MULTIMODAL: 'Aug 26, 2026: Added native support for GLM-5.3-flash, bringing 1M-token context and multimodal input to consumer GPUs.' SPECULATIVE DECODING, CHUNKED PREFILL, DISAGGREGATED P/D and STRUCTURED OUTPUT: none appear in the README, so all four remain UNSTATED and are recorded as such.

Notes

Read 2026-10-04, used to fill the feature-comparison axes for [[ktransformers]]. PREFIX CACHING is stated with its hardware extent, which is the interesting part: 'June 30, 2025: Support 3-layer (GPU-CPU-Disk) prefix cache reuse' - a three-tier prefix cache that spills from GPU to CPU to DISK. That is a different mechanism from every other engine in this repo, whose prefix caches are in-memory only, and it is the natural answer to 'why does this engine exist': CPU/Disk tiers exist because its weights do not fit in VRAM at all, so a KV cache that outlived GPU memory was necessary rather than nice. MULTIMODAL: 'Aug 26, 2026: Added native support for GLM-5.3-flash, bringing 1M-token context and multimodal input to consumer GPUs.' SPECULATIVE DECODING, CHUNKED PREFILL, DISAGGREGATED P/D and STRUCTURED OUTPUT: none appear in the README, so all four remain UNSTATED and are recorded as such.

Cross-references

This record references no other records.