llama.cpp embedding and rerank server paths

COMPUTE-BOUND at batch 1 for a BERT-size encoder, the same structural fact as everywhere else in this slice: one bidirectional attention pass over all input tokens, no KV cache reuse, no autoregressive weight streaming, therefore dense GEMM with arithmetic intensity far above the ~1 flop per byte that makes LLM decode bandwidth-bound. The hardware consequence is the same inversion - for embeddings, buy FLOPS and cache capacity rather than HBM bandwidth. What is distinctive about llama.cpp in this niche is that the encoder workload fits its existing design better than one might expect, for two reasons. First, GGUF removes the whole quantization-tension problem: a 300M encoder in Q8 is ~300 MB and fits anywhere, so the GGUF ecosystem's usual memory-driven motivation largely evaporates and you can keep quantization mild to preserve reranker accuracy. Second, and more interestingly, the encoder path is the rare case where llama.cpp's batch machinery is genuinely useful rather than vestigial: continuous batching and parallel slots exist for decode, but an encoder with no KV cache and uniform per-sequence work is an ideal batching workload, so those knobs transfer cleanly. The one place llama.cpp's decode heritage actively hurts is the KV cache: the server is architected around it and around slot/context accounting, so the token-budgeting knob an encoder deployment actually wants (TEI's --max-batch-tokens) has no analogue here, and an operator has to reason in slots instead. Architectures - can serve: single-vector dense embedding of any BERT-family or GGUF-converted encoder via mean/cls/last pooling; cross-encoder reranking via rank pooling (the flag that exists for exactly this); raw per-token output via --pooling none, which is a building block for ColBERT-style encoders. Cannot serve: SPLADE and other learned-sparse encoders; ColBERT end to end (no MaxSim, no index, no late-interaction storage); multimodal encoders requiring a projector graph; and anything generative on the embedding route. Confidence 0.72: the flag set and the existence of the embeddings and reranking routes are quoted from the server README at master, but llama.cpp's server surface changes weekly and there is no published encoder benchmark, so all throughput claims are absent by design rather than by omission. ||| MAINTENANCE (dated fact, GitHub API, 2026-10-04): last commit 2026-10-04 (0.0 months before this record), latest release v0.5.0 published 2026-09-23, 2077 contributors (distinct committers including anonymous), open_issues_count 2537 (issues PLUS pull requests - see [[ev-github-open-issues-count-includes-pull-requests]]), not archived, 130298 stars. See [[ev-maintenance-status-is-four-dated-facts]] for how to read this fact set. tracing_support=none, verified_negatively=true (set 2026-10-04). What was searched: an in-repo code search for opentelemetry in ggml-org/llama.cpp returns 0 hits; the only observability surface is the hand-built /metrics endpoint assembled by string concatenation in server_task_result_metrics::to_metrics(), which contains no span or exporter code (obs-llamacpp-metrics-source, obs-llamacpp-server-readme-metrics). The absence is repo-level, so the embedding and rerank server paths inherit it.