inference/research
verifiedmedium · 0.80updated 2026-10-03

llama.cpp src/llama-vocab.cpp (master) - token_to_piece and the per-token piece cache

repo

Fields

id
sources/flop3-llamacpp-vocab-source
Publisher
ggml-org/llama.cpp
Kind
repo
Accessed
2026-10-03
notes
Holds cache_token_to_piece, the cache of token id -> text piece, i.e. the table that token_to_piece() consults on every grammar accept and every detokenisation. Establishes that the per-token host cost has a lookup against a vocab-sized cache plus the piece bytes, not a re-tokenisation of the whole sequence.

Notes

Holds cache_token_to_piece, the cache of token id -> text piece, i.e. the table that token_to_piece() consults on every grammar accept and every detokenisation. Establishes that the per-token host cost has a lookup against a vocab-sized cache plus the piece bytes, not a re-tokenisation of the whole sequence.

Cross-references

This record references no other records.