InternVL3 14B

ADDED 2026-10-05 as the missing larger sibling of the existing internvl3-8b record, and to attach a model record to a config source that was already in data/sources/ but cited by no model. GEOMETRY DERIVED FROM config.json (OpenGVLab/InternVL3-14B), read from the llm_config sub-object rather than the top level, because this checkpoint is a multimodal wrapper: llm_config declares architectures [Qwen2ForCausalLM], num_hidden_layers 48, num_attention_heads 40, num_key_value_heads 8, hidden_size 5120, and NO explicit head_dim - so head_dim is derived 5120/40 = 128 and recorded as such. KV CACHE FROM CONFIG: 2 (K and V) x 48 layers x 8 KV heads x 128 head_dim x 2 bytes (bf16) = 2 x 48 x 8 x 128 x 2 = 196,608 B/token (192.0 KiB/token). SCALE CONTRAST WITH THE SIBLING: internvl3-8b is 28 layers x 4 KV heads x 128 x 2 x 2 = 57,344 B/token, so the 14B costs 3.43x the KV per token of the 8B purely because its text tower is Qwen2.5-32B-shaped (48 layers, 8 KV heads) rather than Qwen2.5-7B-shaped (28 layers, 4 KV heads). A reader planning a VLM deployment should not assume the KV scales with the '14B vs 8B' parameter ratio; here it scales at 3.43x, because the vision tower contributes parameters but no KV cache. THE VISION TOWER CACHE IS NOT IN THIS NUMBER and that is deliberate: dynamic_image_size is true and downsample_ratio 0.5, so the number of vision tokens entering the LLM is a function of the tiling at run time, not a config constant. The figure above is per LANGUAGE token; total resident KV per request is this figure times (text tokens + projected image tokens). At 32768 max_position_embeddings the language side alone is 32768 x 196608 = 6.0 GiB. PARAMS: params_b 14 is the published checkpoint size; the safetensors index reports 30.23 GB of tensors, which for a bf14-class dense model with an extra ViT is consistent - the ViT is roughly half the weight count and contributes nothing to the KV figure. CONTEXT SCALING: llm_config declares rope_scaling null, recorded native.