Dynamic resolution makes serving cost unpredictable
[blocker] config
Fields
id
gotchas/dynamic-resolution-makes-serving-cost-unpredictableClass
config
Severity
blocker
Concepts
dynamic-resolutionany-resolutionvision-encoderserving-costattacker-controlled
Symptom
Serving cost is unpredictable because the token count depends on the input image size, which is attacker-controlled. A malicious user can send a 4096x4096 image to cause a 65,536-token prefill, exhausting GPU memory.
Root cause
Dynamic resolution models (NaViT, Pixtral, LLaVA-NeXT/AnyRes) allow images at any resolution, so the token count is a function of the input, not a model constant. This makes the worst-case cost unbounded.
Workaround
Cap maximum image dimensions in the serving config. vLLM and SGLang allow setting max_image_size or similar limits. Alternatively, use a fixed-resolution model or a Perceiver/Q-Former that compresses to a fixed token count.
notes
DYNAMIC RESOLUTION IS A SECURITY RISK: An attacker can send a 4096x4096 image to a Pixtral server, causing a 65,536-token prefill that consumes 20 GB of KV cache (for a 70B model). This can OOM the GPU or cause denial of service. The fix is to cap maximum image dimensions in the serving config. vLLM and SGLang allow setting max_image_size or similar limits. This is analogous to max_tokens for text, but for images. The token counts are derived from the model configs; the security analysis is from the serving cost formula.
Notes
DYNAMIC RESOLUTION IS A SECURITY RISK: An attacker can send a 4096x4096 image to a Pixtral server, causing a 65,536-token prefill that consumes 20 GB of KV cache (for a 70B model). This can OOM the GPU or cause denial of service. The fix is to cap maximum image dimensions in the serving config. vLLM and SGLang allow setting max_image_size or similar limits. This is analogous to max_tokens for text, but for images. The token counts are derived from the model configs; the security analysis is from the serving cost formula.