PaliGemma 3B mix 224

GEOMETRY FILLED 2026-10-17 from the public abetlen text-model Q4_K_M GGUF header [[gguf-paligemma-3b-mix-224-q4km]]. The official config.json is still gated (HTTP 401), so this is a conversion of google/paligemma-3b-mix-224, not Google own file, and it is the text tower only. Parsed: architecture gemma, embedding_length 2048, block_count 18, feed_forward_length 16384, attention.head_count 8, attention.head_count_kv 1, key_length 256, value_length 256, context_length 8192, file_type 15 (Q4_K_M). gqa_ratio = 8/1 = 8, which is MQA. KV at bf16: 18 layers x 1 kv head x 256 head_dim x 2 (K+V) x 2 bytes = 18,432 B/token = 18 KiB/token. active_params_b is set equal to params_b (2.923B, the HF safetensors total) because the text tower is dense; the vision tower is a separate SigLIP encoder with no decode KV. DEPLOYMENT: llama.cpp runs it. The same repo ships the text model at f16, Q8_0 and Q4_K_M plus a separate mmproj-f16 vision projector. vLLM lists PaliGemmaForConditionalGeneration as a native row. No ollama tag was found. The google/paligemma-3b-mix-224 repo returned HTTP 401 on a direct config read this session, so the LLM-tower geometry (hidden size, layer count, head counts) could not be machine-verified here. Left null rather than copied from the widely-repeated PaliGemma 2 3B figures, which are a different checkpoint generation. This is a read-access gap, not an architecture gap - re-reading the config when access is available would close it. TWO-PART MODEL. VERIFICATION IS INCOMPLETE FOR THIS RECORD -- see below. (1) Vision: SigLIP-So400m/14 at 224x224, confirmed from the model card ('The image encoder is initialized from SigLIP-So400m/14'). (2) LLM: Gemma-2B decoder, confirmed from the model card ('initialized from Gemma-2B'). Trained following PaLI-3 recipes. VISION TOKENS = (224/14)^2 = 16*16 = 256 tokens per image at the native 224 resolution -- small and fixed, because PaliGemma's design point is transfer: it is a base model for fine-tuning on captioning, VQA, OCR, detection and segmentation, not a conversational model with a long vision budget. DOMINANCE: roughly balanced; 256 vision tokens sits near a typical short text prompt. ALL LLM DIMENSION FIELDS (hidden_size, num_layers, num_kv_heads, head_dim, gqa_ratio, max_position_embeddings, context_scaling, activation, attention_variant, kv_cache_bytes_per_token) ARE NULL BECAUSE THEY COULD NOT BE VERIFIED, NOT because they are absent from the model. google/paligemma-3b-mix-224 is gated: 'manual' on the HF API and HTTP 401 on raw config.json without an accepted PaliGemma licence. No ungated mirror of this checkpoint was found (unsloth's PaliGemma mirrors also return 401). Per the repo contract these fields are null plus this note rather than filled from the commonly-cited Gemma-2B figures, which would be second-hand paraphrase. params_b 2.923 = HF API safetensors total (2,923,466,480), which IS authoritative and was obtained despite the gate. Note the marketed figure is '3B' and the true count is 2.92B. PaliGemma-2 and PaliGemma-3 checkpoints were NOT recorded: they exist on the Hub but share the same access gate, so nothing beyond their model-card prose would be verifiable. Reference point for the Gemma line: gemma-3-4b-it in this repo has the same SigLIP-27-layer tower at image_size 896, i.e. 4,096 vision tokens versus PaliGemma's 256 at 224 -- a 16x difference purely from input resolution, with no architectural pooling change in between. SERVICEABILITY JOIN (added 2026-10-03; the architecture x engine matrix this record was missing, with a citation per cell in docs/13-model-compatibility.md): FAMILY=multimodal. REQUIRED KERNEL CLASS: an encoder tower (ViT / SigLIP / ConvNet / audio encoder) running on a SEPARATE schedule from the decoder, plus a projector into the decoder's embedding space, plus - for the mrope and DeepStack families - a multi-axis rotary scheme in the decoder rather than plain 1-D RoPE. Three engine-side consequences, all cited. (1) The encoder tower is a separate compute path, so its throughput is a different problem from decode throughput and engines that disclaim encoder throughput are disclaiming this. vLLM says so in its own words: 'We currently support pooling models primarily for convenience. This is not guaranteed to provide any performance improvements over using Hugging Face Transformers or Sentence Transformers directly' [[vllm-pooling-models]] - that is the general statement this class inherits. TRT-LLM's answer is a documented mechanism: a Multimodal Encoder Side Stream that prefetches encoder work on a separate CUDA stream plus a per-model Multimodal Embeddings Cache, currently available only for seven architectures (Gemma4, Gemma4Unified, Mistral3, Qwen3VL, Qwen3VLMoe, Qwen3_5, Qwen3_5Moe) [[mdl2-trtllm-supported-models]]. (2) Several towers carry hard external dependencies: Gemma3nForConditionalGeneration 'depends on timm>=1.0.17 to make use of its MobileNet-v5 vision backbone' and 'is only supported on V1 due to shared KV caching' [[mdl2-vllm-supported-models-2026-10]]; MiniCPM-V 4.6 requires transformers>=5.7.0 because it was upstreamed as a native model type [[mdl2-trtllm-supported-models]]. (3) Hybrid-only multimodal models expose a memory knob unique to this class: vLLM's --language-model-only flag sets all supported multimodal modalities to 0 so the multimodal modules are not loaded, 'to free up more GPU memory for KV cache' for Llama-4, Step3, Mistral-3 and Qwen-3.5 [[mdl2-vllm-supported-models-2026-10]].SUPPORTED BY (cited): vLLM native rows include InternVLChatModel and InternVLForConditionalGeneration, LlavaOnevisionForConditionalGeneration, LlavaNextForConditionalGeneration, MolmoForCausalLM and Molmo2ForConditionalGeneration, PaliGemmaForConditionalGeneration, SmolVLMForConditionalGeneration, Glm4vForConditionalGeneration, Qwen2VL/Qwen3VL, MiniCPMV, Mllama, Phi4MM, Gemma4, NemotronH_Nano_VL_V2, VoxtralForConditionalGeneration (which requires mistral-common[audio]) and Qwen3OmniMoeThinkerForConditionalGeneration [[mdl2-vllm-supported-models-2026-10]]. SGLang's multimodal table covers InternLM2-VL, LLaVA-OneVision, LLaVA-NeXT(-Video), Ling-3.0-flash-VL, LFM2.5-VL, MiniCPM-V-2, GLM-4.5V, GLM-OCR, Qwen3-VL 235B-A22B and 30B-A3B, Qwen3-Omni-30B-A3B, Kimi-VL-A3B, MiMo-VL-7B-RL, Step3-VL-10B, dots.ocr, PaddleOCR-VL, Cosmos3, deepseek-vl2, Janus-Pro-7B, NVILA-8B and gemma-3-4b-it, plus openai/whisper-large-v3 and Qwen3-ASR-1.7B on the audio route [[mdl2-sglang-multimodal-language-models]]. TRT-LLM's multimodal matrix covers a narrower set with an explicit per-row modality string and per-row Untested cells [[mdl2-trtllm-supported-models]]. Older GGUF-side engines reach the class through llama.cpp's projector graph, which llama.cpp-embedding-server states it cannot serve on the embedding route [[llama-cpp-embedding-server]]. PER-MODALITY CAVEATS CITED: TRT-LLM marks Gemma4Unified and MiniCPM-V 4.6 Untested for CUDA Graph / chunked prefill, MiniMax-M3 Untested for attention DP and KV cache reuse, Qwen4Exp Untested across the board, and notes audio requires a checkpoint with a sound_config and is supported only on the full non-disaggregated path [[mdl2-trtllm-supported-models]]. Step-3.7's vision tower runs in BF16 even when the text decoder is quantized [[mdl2-trtllm-supported-models]] - a precision asymmetry that silently costs memory. vLLM's fallback path accepts image inputs only, with video 'to be added in a future release' [[mdl2-vllm-supported-models-2026-10]]. CONTEXT_SCALING_METHOD (none-verified): google/paligemma-3b-mix-224 config.json is gated (401); record has no max_position_embeddings and no attention_variant.