# ShieldGemma 2 ShieldGemma 2 is Google's open image safety classifier: 4B parameters, built on Gemma 3. You hand it an image and a written safety policy, it tells you whether the image violates that policy. It works on both real photographs and generated images, which is the point. If you ship anything that produces or accepts images, you need a moderation layer, and building one from scratch is a project nobody wants. ## How it works The prompt sandwiches your policy between `<BEGIN_SAFETY_POLICY>` and `<END_SAFETY_POLICY>` tags, then asks: does the image violate the above policy? The model returns probabilities for the `Yes` and `No` tokens. A high `Yes` means a violation. You pick the threshold, which means you control how strict the filter is instead of accepting someone else's calibration. Three policies ship by default: | Policy | Covers | |---|---| | Sexually Explicit Content | Graphic sexual depictions | | Dangerous Content | Weapons, explosives, self-harm, anything facilitating real-world harm | | Violent Content | Shocking, sensational or gratuitous violence | Reported precision / recall / F1 on Google's internal sets: 87.6 / 89.7 / 88.6 for sexually explicit, 95.6 / 91.9 / 93.7 for dangerous content, 80.3 / 90.4 / 85.0 for violence and gore. ## The limitation worth knowing The model is highly sensitive to how you word the policy. Change the description and the classification changes with it. That cuts both ways. It's genuinely flexible, since you can write a policy for your own product rather than accepting three fixed categories. But it also means the policy text is now part of your system, and it needs versioning, testing and review like any other piece of logic. Treat it as code, not configuration. Nuance and ambiguity remain hard. This is a filter, not a judge. Anything consequential still needs a human. ## Where it fits Input filtering on user uploads, output filtering on generated images, or both. See [[AI Guardrails]] for the broader pattern, and note that a 4B classifier is small enough to sit inline without wrecking your latency budget. Available on Kaggle and [[HuggingFace]] under the Gemma Terms of Use. ## References - Model card: https://ai.google.dev/gemma/docs/shieldgemma/model_card_2 - ShieldGemma documentation: https://ai.google.dev/gemma/docs/shieldgemma - Hugging Face collection: https://huggingface.co/collections/google/shieldgemma-2 - Gemma documentation: https://ai.google.dev/gemma/docs ## Related - [[Gemma]] - [[Gemma 4]] - [[Google]] - [[Google DeepMind]] - [[AI Safety]] - [[AI Guardrails]] - [[HuggingFace]] - [[Gemma AI Skills]] - [[EmbeddingGemma]]