inference/research
verifiedmedium · 0.80updated 2026-10-04

Quest (query-aware KV page sparsity) - NOT KV quantization

none - no bits reduced. Only the top-K critical KV pages are LOADED for attention; the rest stay in memory but are not read.

Fields

id
quantization/quest-kv-sparsity
Scheme
kv-sparsity
Bits
none - no bits reduced. Only the top-K critical KV pages are LOADED for attention; the rest stay in memory but are not read.
Activation scheme
None. Page-level min and max Key values are tracked in the KV cache pages; a query vector estimates each page's criticality and only the top-K pages are fetched for attention. Sparsity, not quantization.
Kernels
Quest implementation (mit-han-lab/Quest); designed to reduce the KV-cache LOADING cost that dominates attention at long context.
Quality delta
Measured quality from Tang et al. (ICML 2024, arXiv:2406.10774, qlab-quest-paper): On PG19 32k-context language modeling (LongChat-7B-v1.5-32k), Quest with token budget 4096 (1/8 cache) matches full-cache oracle perplexity within 0.01 ppl (Fig. 3 and Fig. 6). On 10k passkey retrieval (Table 1), Quest achieves 99% accuracy with budget 128 and 100% with budget 512 (vs H2O 1%, TOVA 1%, StreamingLLM 5%). On 100k passkey retrieval (Yarn-Llama-2-7B-128k), Quest achieves 96% with budget 1024 and 100% with budget 2048 (vs H2O 4%, TOVA 10%, StreamingLLM 4%). On LongBench, achieves lossless task performance with 1/5 to 1/10 KV cache budgets.
notes
GROUP SIZE NOT APPLICABLE BY CONSTRUCTION: page/head-level selection of which KV to keep or prefetch. No weights are re-gridded. Recorded to mark the boundary again, and it is a sharp one: Quest is the method that most clearly shows why KV quantization and KV sparsity are different tools. Quest's insight is that token criticality is QUERY-DEPENDENT - a fixed eviction policy like H2O or SnapKV commits in advance to which tokens matter, while Quest picks per query and therefore keeps information that a static policy would have discarded. Consequence for a serving system: quantization is unconditional and always applies, sparsity is selective and only pays off at long context. Both are recorded in the KV family because both answer 'does this request fit', which is why this problem matters as much as weight quantization. SCHEME RECLASSIFIED (parent agent, 2026-10-03): scheme was 'spinquant' (a proxy recorded while the enum lacked a legal value) and is now 'kv-sparsity'. NATIVE_SUPPORT CLASSIFIED 2026-10-04: this field is still [] and that is CORRECT, but the empty list used to be ambiguous between three different states and now is not. Reason class on this record: KV-CACHE-ONLY, SO STRUCTURALLY ABSENT. The bits field records that no bits-per-value change: the scheme reduces WHICH positions are retained, fetched or reallocated, so it runs on whatever precision path the accelerator already has for the KV cache. A KV-position policy can never be a hardware format. Consequence for a reader: nothing further to verify - this is a cache-management policy, not a numeric format. The full statement of the three-way distinction, and the worked list of all 72 empty records by class, is in gotcha quant-native-support-empty-means-three-different-things. Rule of thumb for this field everywhere in the repo: if the bits field carries an 'a16', or the scheme is KV-cache-only or a llama.cpp integer storage type, an empty native_support is a DEFINITION and needs no verification; only a W4A4/W8A8/sub-8-bit format with research kernels can have an empty list that is a real finding.

Notes

GROUP SIZE NOT APPLICABLE BY CONSTRUCTION: page/head-level selection of which KV to keep or prefetch. No weights are re-gridded. Recorded to mark the boundary again, and it is a sharp one: Quest is the method that most clearly shows why KV quantization and KV sparsity are different tools. Quest's insight is that token criticality is QUERY-DEPENDENT - a fixed eviction policy like H2O or SnapKV commits in advance to which tokens matter, while Quest picks per query and therefore keeps information that a static policy would have discarded. Consequence for a serving system: quantization is unconditional and always applies, sparsity is selective and only pays off at long context. Both are recorded in the KV family because both answer 'does this request fit', which is why this problem matters as much as weight quantization. SCHEME RECLASSIFIED (parent agent, 2026-10-03): scheme was 'spinquant' (a proxy recorded while the enum lacked a legal value) and is now 'kv-sparsity'. NATIVE_SUPPORT CLASSIFIED 2026-10-04: this field is still [] and that is CORRECT, but the empty list used to be ambiguous between three different states and now is not. Reason class on this record: KV-CACHE-ONLY, SO STRUCTURALLY ABSENT. The bits field records that no bits-per-value change: the scheme reduces WHICH positions are retained, fetched or reallocated, so it runs on whatever precision path the accelerator already has for the KV cache. A KV-position policy can never be a hardware format. Consequence for a reader: nothing further to verify - this is a cache-management policy, not a numeric format. The full statement of the three-way distinction, and the worked list of all 72 empty records by class, is in gotcha quant-native-support-empty-means-three-different-things. Rule of thumb for this field everywhere in the repo: if the bits field carries an 'a16', or the scheme is KV-cache-only or a llama.cpp integer storage type, an empty native_support is a DEFINITION and needs no verification; only a W4A4/W8A8/sub-8-bit format with research kernels can have an empty list that is a real finding.

Cross-references

Emulated support (5)