Llama Guard 3-1B-INT4: Compact and Efficient Safeguard for Human-AI Conversations

LLM-based guardrails are a SECOND MODEL ON THE CRITICAL PATH of every conversation, so their size and latency are a direct tax on every interaction - and the 8B-class guard models are too heavy to deploy on constrained or edge devices.

Two compounding changes to the same guardrail design as Llama Guard (pap-eff-llama-guard-safeguard-model): shrink the guard model to the 1B class AND quantise it to INT4, yielding a 440MB artifact. The int4 quantisation is what makes it run on a commodity ARM CPU at all - the deployment target is a phone-class device, not a datacenter GPU, so the entire cost argument is about fitting the guardrail on hardware that already exists at the edge.

At least 30 TOKENS PER SECOND throughput and time-to-first-token of 2.5 SECONDS OR LESS ON A COMMODITY ANDROID MOBILE CPU. Attains COMPARABLE OR SUPERIOR safety moderation scores to its larger counterpart Llama Guard 3-1B, despite being approximately 7 TIMES SMALLER in size (440MB). This is one of the very few records in the entire set that quotes a concrete tok/s and TTFT for a deployed model on named consumer hardware, which is why hardware_relevance is deliberately left empty rather than padded with a datacenter id it does not use.

ADOPTION: in-production - open-sourced at Meta Connect 2024 (per the abstract) with released weights (open_weights true) and intended for resource-constrained deployment. VENUE NOTE: no peer-reviewed venue verifiable via Semantic Scholar and no arXiv comment venue, so venue = arxiv-preprint, year = ArXiv v1 (Nov 2024). THE RECORD-LEVEL CONTRIBUTION IS THE COST ARITHMETIC, and it is the most quotable guardrail number in the repo: it establishes that a safety check can be had for 2.5s TTFT and 30 tok/s on a commodity mobile CPU at 440MB, which reframes 'guardrails are too expensive to run per-request' from a design constraint into an engineering tradeoff with a measured price. The 'comparable or superior' moderation claim is against the 1B non-quantised counterpart on safety scores - NOT against the 8B guard, and a reader who concludes int4 1B matches the 8B model would be over-reading. Two sizes are in play in the comparison and the record deliberately keeps them distinct. AFFILIATIONS FILLED 2026-10-04 (paper-verification pass): from the Semantic Scholar graph API (paper/arXiv:2411.17713), which is the only machine-readable source that carries author affiliations — the arXiv Atom feed returns author names only, and no ORCID is exposed by either API. One known upstream typo corrected: "Tel Aviv Univesity" -> "Tel Aviv University". S2 affiliation coverage is partial (120 of 249 corpus records carry any), so these lists may be incomplete rather than exhaustive. VENUE_TRACK BACKFILL 2026-10-04 (paper-fields pass, agent w3p): venue_track 'unknown'. `venue` is 'arxiv-preprint', which means no peer-reviewed venue has been established for this work as of the verification date, so the track genuinely is not known. 'unknown' is used here rather than leaving the field null or guessing 'main': the schema separates 'unknown' from 'main' precisely so that an unprereviewed preprint is not silently counted as main-track work. This annotation is what lets a downstream filter distinguish 'peer reviewed, main track' from 'not peer reviewed' without inferring it from the venue string.