Models

"Every production LLM has a tiny chaperone model riding shotgun, distilled small enough to fit the latency budget and bored enough to flag itself."

Distill, Sub-100ms Guardrail AI Agent

Models for responsible AI fall into two operational positions: inline models that run alongside the main LLM at request and response time (Llama Guard scanning every output) and offline models that score logged data after the fact (Detoxify running over yesterday's logs to surface incidents). Picking right matters because the constraints differ: inline models must fit a latency budget (sub-100ms is the target for most user-facing applications) and a cost budget (you pay for two model calls per request); offline models can be larger and slower but must keep up with traffic in batches. The 2024-26 trend is small distilled guards (Llama Guard 4 at 12B, ShieldGemma at 2B / 9B / 27B, Granite Guardian variants from 2B to 13B) that fit inline, paired with larger generalists for hard adjudication.

56.4.1 Safety classifiers

The fastest way to get fired in production AI is to ship a model that helps a teenager write a suicide note or guides a user through synthesizing fentanyl. Safety classifiers are the cheap, small, always-on chaperones that sit between your main LLM and the user, scoring both prompts and responses across harm categories (violent, sexual, hate, self-harm, criminal advice, prompt injection, jailbreak, child safety). They are the inline policy layer, and in 2026 they are how every serious deployment keeps the catastrophic 1-in-100,000 failure off the front page.

• Llama Guard 3 and Llama Guard 4 (Meta, 2024-2025) are the canonical open-weights safety classifiers, built by fine-tuning Llama base models on a curated taxonomy of 14 harm categories aligned with the MLCommons AILuminate hazard taxonomy. Their objective is to be the production-grade open-weights guard that any LLM application can drop in front of any LLM backend, which matters because the closed-API moderation endpoints (OpenAI Moderation, Azure AI Content Safety) lock you to a vendor. The core concept is the prompt-format-instructed classifier: a single forward pass labels a (user, assistant) message pair as "safe" or "unsafe" with category tags. Pick Llama Guard 3 / 4 as the default self-hosted guard in 2026; pair with NeMo Guardrails or LLM Guard as the orchestration layer.
• Granite Guardian 3.x (IBM, 2024-2025) is IBM's open-weights safety-classifier family, distinguished by explicit support for RAG-context risks (groundedness, context relevance, answer relevance, jailbreak) alongside the standard harm categories. Its objective is to be the guard tuned for enterprise RAG applications where "did the model fabricate something not in the context?" is as important as "did the model say something harmful?", which matters because RAG-specific failures are a primary production complaint. The core concept is the multi-task guardrail: one model scores both safety violations and RAG-specific risks. Pick Granite Guardian when RAG-specific risks (groundedness, hallucination) need first-class guardrails; for pure safety-policy guarding, Llama Guard is the more focused choice.
• ShieldGemma (Google, 2024) is Google's open-weights safety-classifier family built on Gemma 2, available in 2B / 9B / 27B sizes for different latency budgets. Its objective is to provide a Gemma-family safety guard with the same licensing terms as Gemma itself, which matters when you have already committed to the Gemma stack. The core concept is per-category classifiers (dangerous content, harassment, hate speech, sexually explicit) plus an aggregate harm score. Pick ShieldGemma when Gemma is the base model; for non-Google-ecosystem teams Llama Guard has wider community support.
• OpenAI Moderation API (omni-moderation-2024-09-26) (OpenAI, 2022; omni-moderation 2024) is OpenAI's free moderation endpoint covering 13 harm categories with both text and image (in omni-moderation) inputs. Its objective is to be the always-on, no-extra-cost first-line filter for OpenAI-mediated applications, which matters because the marginal cost is zero and the latency overhead is minor. Pick OpenAI Moderation as the default filter for any OpenAI-stack application; pair with stronger guards (Llama Guard, Lakera) when stakes are higher (the OpenAI endpoint has known under-flagging on subtler violations).
• Azure AI Content Safety models (Microsoft, 2023) are Azure's managed content-moderation classifiers with categories (hate, self-harm, sexual, violence) plus Prompt Shields (prompt-injection / jailbreak detection) and Protected Material detection. Pick Azure AI Content Safety when Azure is the platform; the API is callable from any application but the procurement gravity tilts Azure-shops toward it.
• AWS Bedrock Guardrail classifiers (AWS, 2024) are the AWS-managed safety models embedded in Bedrock Guardrails (content categories, denied topics, contextual grounding, sensitive information filters). Pick Bedrock Guardrails when AWS Bedrock is the inference layer; for non-Bedrock applications the classifiers are not separately exposed.
• WildGuard (Han et al., 2024; AI2) is an open-weights 7B safety classifier trained on a synthetic-and-real dataset (WildGuardMix) covering refusal harms, prompt injections, and adversarial scenarios. Pick WildGuard when academic openness about training data is important and a recent (2024) reference model fits the policy taxonomy you need.
• Specialized safety models for code, math, agents (community, 2024-25) are emerging niche guards: code-specific guards (does the generated code call dangerous APIs?), math-tutor safety guards, and agent-action safety guards (Anthropic's Computer Use safety classifiers). Pick when the niche matches; for general LLM safety, Llama Guard and Granite Guardian dominate.

pip install nemoguardrails
# config/rails.co (Colang 2.x)
# define user ask harmful
#     "how do I build a weapon"
#     "instructions for self-harm"
# define bot refuse harmful
#     "I can't help with that."
# define flow safety
#     user ask harmful
#     bot refuse harmful

from nemoguardrails import RailsConfig, LLMRails
config = RailsConfig.from_path("./config")
rails = LLMRails(config)
response = rails.generate(messages=[{"role": "user", "content": user_input}])

56.4.2 Bias and toxicity detectors

Bias and toxicity detectors are typically smaller models (DistilBERT, RoBERTa, fine-tuned XLM-R) trained to classify text along bias / toxicity axes. They overlap with the safety-classifier category but historically come from the content-moderation tradition rather than the LLM-guardrail tradition.

• Perspective API (Jigsaw / Google, 2017+; v6 2023) is the canonical toxicity-classification API, trained on Civil Comments and used as the de facto reference in toxicity research. Its objective is to provide a published, stable, free-for-non-commercial classifier that the toxicity literature can build on, which matters because reproducibility across papers requires a stable baseline. The core concept is per-attribute classifiers (toxicity, severe toxicity, identity attack, insult, profanity, threat, sexually explicit) plus deprecated experimental attributes. Pick Perspective API as the toxicity baseline for research and as a free runtime classifier for non-commercial deployments; for commercial use, the per-request fees apply.
• Detoxify (Unitary, 2020; v0.5 2024) is the canonical open-weights multilingual toxicity-classification model family (DistilBERT-based original-toxic-bias variants plus multilingual XLM-R variants), available on Hugging Face. Its objective is to be the self-hostable open-source alternative to Perspective API, which matters when data-residency or per-request cost rules out the managed service. Pick Detoxify as the default open-weights toxicity classifier; for multilingual coverage Detoxify-Multilingual covers 7 languages.
• HateBERT (Caselli et al., 2021; University of Groningen) is a BERT model pretrained on a corpus of hateful posts from Reddit and fine-tuned for abusive-language detection. Its objective is to specialize the language-model substrate for hate-detection, which matters because BERT trained on general corpora misses some hate-speech vocabulary. Pick HateBERT for hate-speech-specific classification, especially when the deployment domain matches Reddit-style discourse; for general toxicity, Detoxify is more flexible.
• ToxiGen-RoBERTa (Hartvigsen et al., 2022; Microsoft) is a RoBERTa model trained on ToxiGen (Section 56.3) to detect implicit / adversarial hate that escapes overt-slur detectors. Pick ToxiGen-RoBERTa when implicit hate is the failure mode (Detoxify catches overt; ToxiGen-RoBERTa catches subtler).
• HONEST (Hurtful Open-ended ENglish-language Sentence Templates) (Nozza et al., 2021; Bocconi) is a benchmark and accompanying scoring method for measuring hurtful completions in LMs across protected groups, distinguished by template-based probes designed to reveal completion bias. Pick HONEST for measuring LM completion bias; pair with BBQ and BOLD for fuller LM-bias coverage.
• RoBERTa-Hate-Speech-Dynabench (Vidgen et al., 2021; Meta + Dynabench) is the dynamic-benchmark-trained hate-speech classifier from the Dynabench platform, distinguished by being trained against adversarial human-in-the-loop attacks. Pick this model when adversarial robustness in hate detection matters; for static-data trained classifiers, Detoxify or HateBERT.
• SafeNLP and biased-association detectors (community) are smaller open models for specific bias-measurement tasks (gender-occupation association, word-embedding-association tests). Pick when measuring specific embedding-association biases (e.g., the WEAT/SEAT test family); for general detection, larger classifiers dominate.

56.4.3 Watermark and AI-content detectors

AI-content detectors split into watermark detectors (which read an embedded signal) and detection-by-statistics models (which try to distinguish AI-generated text from human-written text using statistical fingerprints). The two categories have very different reliability profiles.

• SynthID-Text detector (Google DeepMind, 2024) is the verifier-side companion to the SynthID-Text watermarker (Section 56.2), distinguished by detecting watermarks Google embeds in Gemini outputs and (via Hugging Face's open-sourced detector) any model that has been configured to emit SynthID watermarks. Its objective is to verify provenance of text without false-positives on human writing, which matters because AI-detection has a well-documented false-positive problem on non-native-English authors. Pick SynthID-Text detection when the source model is known to emit SynthID watermarks; for arbitrary AI-generated text where no watermark was applied, statistical detectors are the alternative with worse reliability.
• OpenAI AI Text Classifier (retired Jan 2023) (OpenAI, 2023) was OpenAI's short-lived AI-written-text classifier, withdrawn in July 2023 due to low accuracy and a high false-positive rate on non-native-English writing. Its inclusion here is historical: the lessons from its retirement (statistical AI-detection is unreliable in the wild, false positives disproportionately affect non-native speakers) shaped the field's pivot to watermark-based detection. Do not deploy this retired classifier; the reference is for understanding the historical pivot.
• Originality.ai (Originality.ai, 2022) is a commercial AI-content-detection service with classifiers tuned for detecting GPT-family generation and (more recently) Claude and Gemini generation. Its objective is to be the academic-integrity and content-marketing detector marketed at writers and editors. Pick Originality.ai when a managed detector with vendor support is the requirement; treat results as advisory rather than definitive (statistical AI-detection is fundamentally limited).
• GPTZero (GPTZero, 2023) is the academic-integrity-focused AI-text detector that became widely known in 2023, distinguished by classroom-targeted UX. Pick GPTZero for K-12 / higher-education contexts; the underlying classifier has the same fundamental limitations as other statistical detectors.
• DetectGPT and Fast-DetectGPT (Mitchell et al., 2023; Stanford) are zero-shot AI-detection methods based on perturbation-curvature analysis (AI-generated text occupies sharper minima of the log-probability landscape than human text). Their objective is to detect AI generation without training a classifier, which matters for new models where labeled training data does not exist. Pick DetectGPT-family for research benchmarks; for production, the false-positive problem is the same as other statistical methods.
• Binoculars (Hans et al., 2024) is a 2024 zero-shot AI-detection method using cross-perplexity between two related LMs, distinguished by claimed lower false-positive rates than DetectGPT. Pick Binoculars for the current state-of-the-art zero-shot statistical detector; benchmark on your distribution before deploying.
• Turnitin AI Detection (Turnitin, 2023) is the AI-content-detection module bolted onto Turnitin's plagiarism-detection platform, distinguished by integration with university LMSs. Pick Turnitin when university-LMS integration is the deciding factor; the underlying classifier shares the field's reliability issues.

56.4.4 Aligned and constitutional base models

Aligned base models ship with safety as a first-class property: trained with RLHF or constitutional methods, paired with safety classifiers, documented with model cards and red-team reports. They are the "models you should deploy as the LLM in your application" when responsibility is a constraint.

• Claude family (Opus 4.5, Sonnet 4.5, Haiku 4.5) (Anthropic, 2023-2026) are the canonical "trained-for-helpfulness-honesty-harmlessness" frontier models, distinguished by Constitutional AI training (the model critiques and revises its own outputs against a written constitution) and public Responsible Scaling Policy commitments tying capability levels to safety evaluations. Their objective is to make safety a property of the base model rather than only a wrapper, which matters because guardrails alone cannot fix a poorly aligned base. The core concept is RLAIF + Constitutional AI: the model is trained to follow principles articulated in the constitution rather than only labeled preferences from human raters. Pick Claude as a frontier default when responsible-AI is the constraint; ship Llama Guard or Anthropic's safety-evaluation suite alongside as defense-in-depth.
• Llama-3.x and Llama 4 with Llama Guard (Meta, 2024-2025) are the canonical open-weights aligned family, distinguished by published model cards covering safety evaluations, the Llama Guard companion (same section), and an RLHF + DPO recipe that ships fine-tunable. Their objective is to give the open-weights community a frontier-adjacent model with a documented safety story, which matters when you self-host. Pick Llama-3.x or Llama-4 + Llama Guard as the canonical open-weights aligned stack; the Llama Guard companion is mandatory not optional in production.
• OLMo with Dolma training-data attributions (AI2, 2024-2025) are AI2's fully-open language models (open weights and open training data via Dolma), distinguished by training-data transparency: every training example is published, so users can inspect what the model learned from. Their objective is to be the model that supports the strongest scientific reproducibility, which matters for academic research and for governance audits that require training-data inspection. Pick OLMo when training-data transparency is a requirement (regulator-mandated lineage, copyright-conscious deployments); for raw capability the frontier closed models lead.
• Gemma 2 / 3 with ShieldGemma (Google, 2024-2025) are Google's open-weights aligned-and-instruction-tuned family, distinguished by the ShieldGemma companion and tight integration with the Vertex AI safety toolchain. Pick Gemma 2 / 3 + ShieldGemma when you want a Google-ecosystem open-weights alignment stack; for non-Google ecosystems, Llama + Llama Guard is the analog.
• Qwen family with safety fine-tunes (Alibaba, 2024-2025) are Alibaba's open-weights models with explicit safety fine-tunes and a published safety-evaluation report. Pick Qwen for Chinese-language safety alignment and for cost-efficient open-weights deployment; for English-language ecosystems with broader tooling, Llama remains more conventional.
• Mistral family (Mistral AI, 2023-2025) are open-weights European-headquartered models with mid-2024 alignment-and-safety updates. Pick Mistral when European AI-policy alignment (EU AI Act readiness, European HQ procurement) is the constraint; pair with external guards (Llama Guard, LLM Guard) as Mistral itself has not shipped a first-party guard model.
• Claude Haiku 4.5 specifically for low-cost inline scoring (Anthropic, 2025-2026) is sometimes deployed as an LLM-as-judge for safety scoring when latency permits, distinguished by being a frontier-trained model used cheaply for narrow tasks. Pick Haiku as an LLM-judge in safety evaluation when small-classifier models (Llama Guard, ShieldGemma) miss niche violations; the cost is higher per call but the policy coverage is broader.

56.4.5 Interpretability-oriented models

Interpretability-oriented models exist specifically to support mechanistic-interpretability and circuit-level safety research. They are usually smaller than frontier production models but trained with extra instrumentation, sparse autoencoder features, or open activations.

• Gemma-Scope (Google DeepMind, 2024) is a public library of sparse autoencoders (SAEs) trained on every layer of Gemma 2 (2B and 9B), distinguished by being the largest public release of SAE features at the time. Its objective is to give the mechanistic-interpretability community a frontier-class model with publicly-available learned feature dictionaries, which matters because SAE training is expensive and only one of the major labs has been able to publish frontier-scale SAEs. The core concept is the per-layer SAE: each Gemma-2 layer activation is decomposed into a sparse combination of human-interpretable features. Pick Gemma-Scope as the default starting point for SAE-based mechanistic-interpretability work; pair with TransformerLens (Section 56.2) for the analysis tooling.
• Anthropic Crosscoder and SAE models (Anthropic, 2024-2025) are Anthropic's sparse autoencoder releases on Claude 3 Sonnet and on toy models for the "Scaling Monosemanticity" research line. Their objective is to scale SAE-based feature extraction to frontier-class models, which matters because most published mechanistic-interpretability work pre-2024 was on small toy models. Pick the Anthropic SAE releases (limited public availability) for SAE-on-Claude research; for routinely-accessible models, Gemma-Scope is the public alternative.
• Pythia and the Interpretability Research Models (EleutherAI, 2023) are a suite of language models (70M-12B parameters) released with full training trajectories and checkpoints, designed specifically for interpretability research. Their objective is to provide an open scientific instrument for studying how language models change during training, which matters when you need access to checkpoint-by-checkpoint training dynamics. Pick Pythia for training-dynamics research; for capability frontier work, frontier open-weights models are stronger.
• SAE Lens releases for Llama / Mistral / Gemma (Decode Research / community, 2024) are community-trained sparse autoencoders on open-weights frontier models, distributed via the SAE Lens library. Pick SAE Lens releases when you want to extend Gemma-Scope-style analysis to non-Gemma models; quality varies by SAE provenance.
• GPT-2 with Anthropic / Conjecture SAE releases (Anthropic, 2022; community 2023+) are the historical reference: SAE-trained GPT-2 models from the "Toy Models of Superposition" line. Pick when teaching SAE methodology (the GPT-2 scale is small enough for fast iteration); for serious research, larger models with newer SAEs are more representative.
• TransformerLens demo models (HookedTransformer) (Nanda et al., 2022+) are not models per se but TransformerLens-loaded versions of common open-weights models (GPT-2, Pythia, Llama-family, Gemma) with activation hooks already wired in. Pick when starting mechanistic-interpretability work on any of these base models; the library does the model-wrapping for you.
• SAEBench and SAE evaluation suites (Decode Research, 2024) are evaluation benchmarks for sparse autoencoders themselves (sparsity, reconstruction, interpretability metrics). Pick SAEBench when training or comparing SAEs; for downstream use of SAEs, Gemma-Scope is ready-to-use.

56.4.6 Models by deployment position

56.4.7 A canonical 2026 model stack

56.4.8 Licensing and deployment considerations

• Llama Guard 3 / 4 and Llama base models: Llama Community License, which is permissive for most uses but has acceptable-use restrictions and large-deployment terms. Inspect the license for your specific deployment.
• Granite Guardian: Apache 2.0; IBM is unusually permissive with the Granite family.
• ShieldGemma and Gemma 2/3: Gemma Terms of Use (similar to Llama Community License with usage restrictions).
• OpenAI Moderation API, Azure AI Content Safety, AWS Bedrock Guardrails: vendor terms; OpenAI Moderation is free, the cloud guards are paid per request.
• Perspective API: free for non-commercial use, paid commercial tiers; Jigsaw policies apply.
• Detoxify, HateBERT, ToxiGen-RoBERTa: Apache 2.0 or MIT; freely usable.
• Claude family: API-only; Anthropic's usage policies apply (no model-weights download).
• OLMo + Dolma: Apache 2.0 and ODC-BY for the dataset; the open-data license is a distinctive feature.
• Gemma-Scope: Apache 2.0 for the SAEs; subject to the underlying Gemma license for the base model.
• Pythia and SAE Lens releases: Apache 2.0 / MIT.
• Originality.ai, GPTZero, Turnitin AI Detection: vendor SaaS terms; per-request or seat-based pricing.

56.4.9 Model evaluation checklist

The questions to ask when adopting a responsible-AI model (safety classifier, bias detector, watermark, aligned base, interpretability release):

• Training data transparency: is the training data published (OLMo + Dolma) or only described in a model card? Closed training data limits audit ability and constrains regulator acceptance.
• Taxonomy alignment: does the model's harm taxonomy align with your policy taxonomy and with industry-standard taxonomies (MLCommons AILuminate)? A model trained on a different taxonomy will produce labels that do not map cleanly to your categories.
• Latency at your batch size: profile the model at the batch size and sequence length you will deploy. A 12B safety classifier that hits 80ms at batch size 1 may hit 200ms at the batch size 8 you actually run.
• Refusal-rate calibration: how often does the model refuse benign inputs (false-positive refusal)? Aligned base models with high refusal rates may make legitimate use cases impossible.
• Multilingual coverage: does the model work in your deployment's languages? Many safety classifiers were trained on English and degrade meaningfully on other languages.
• Update cadence: how often does the vendor or community release updated weights? Stale models miss newer attack patterns and shifting policy norms.
• License compatibility: does the model's license (Llama Community License, Gemma Terms of Use, Apache 2.0, OpenRAIL) permit your deployment (commercial, restricted-use, derivative work)?
• Eval-result reproducibility: can you reproduce the published benchmark numbers locally? If not, treat the published numbers as marketing rather than evidence.
• Failure mode documentation: does the model card list known failure modes and over-refusal patterns? An undocumented model is harder to integrate safely.
• Adversarial robustness: has the model been red-teamed against the attack patterns you will face (prompt injection, jailbreak, character-rewrap, multilingual evasion)?

A team that asks these questions during evaluation usually picks a different model than a team that picks based on the published benchmark numbers alone.