Llama Guard 3-1B-INT4: Compact and Efficient Safeguard for Human-AI Conversations
paper
Fields
id
sources/sec-arxiv-2411-17713Publisher
Quantised 440 MB guard (about 7x smaller than Llama Guard 3-1B) reaching at least 30 tokens/s and TTFT of 2.5 s or less on a commodity Android mobile CPU, with comparable or superior moderation scores. Useful as the LOW BOUND on guard-model cost: even the smallest published guard adds seconds, not milliseconds.
Kind
paper
Published
2024-11-18
Accessed
2026-10-04
Cross-references
This record references no other records.