inference/research
verifiedhigh · 0.85updated 2026-10-03

SGLang issue #39147: HiCacheFile reports an unrestorable prefix for hybrid cache pools

forum

Fields

id
sources/pd-gh-sglang-39147-hicache-false-prefix-hit
Publisher
sgl-project/sglang
Kind
forum
Published
2026-09-11
Accessed
2026-10-03
notes
Correctly scoped by its own author, which is why it is usable: 'This report establishes an incorrect backend existence-query result. It does not claim a reproduced model-output error or an end-to-end load-back failure.' HiCacheFile.batch_exists_v2() takes each pool's longest usable prefix and the MINIMUM across pools, which is wrong for TRAILING_PAGES pools that can have holes in their valid endpoints: with SWA endpoints {3} and Mamba endpoints {2} it reports 2 (expected 0), and with SWA {1,3} and Mamba {1,2} it reports 2 (expected 1, overlooking an earlier valid common checkpoint). 'In the latter two cases, the reported endpoint has no SWA object.' Fix direction: intersect legal endpoints, as MooncakeStore.batch_exists_v2() already does. Relevance: a KV-tier existence query that over-reports is the precondition for loading a prefix that cannot actually be served.

Notes

Correctly scoped by its own author, which is why it is usable: 'This report establishes an incorrect backend existence-query result. It does not claim a reproduced model-output error or an end-to-end load-back failure.' HiCacheFile.batch_exists_v2() takes each pool's longest usable prefix and the MINIMUM across pools, which is wrong for TRAILING_PAGES pools that can have holes in their valid endpoints: with SWA endpoints {3} and Mamba endpoints {2} it reports 2 (expected 0), and with SWA {1,3} and Mamba {1,2} it reports 2 (expected 1, overlooking an earlier valid common checkpoint). 'In the latter two cases, the reported endpoint has no SWA object.' Fix direction: intersect legal endpoints, as MooncakeStore.batch_exists_v2() already does. Relevance: a KV-tier existence query that over-reports is the precondition for loading a prefix that cannot actually be served.

Cross-references

This record references no other records.