Content filters for prompts and outputs

Problem

GenAI agents must resist prompt injection, avoid harmful outputs, and maintain provenance. Azure AI Content Safety offers moderation for text/images and safety features (Prompt Shields, groundedness detection, protected material) with configurable thresholds and blocklists.

Options and trade-offs

• Prompt Shields (direct and indirect injection): Detect jailbreak attempts and document-borne attacks; low false-positive rates; add latency; configurable block vs annotate modes.
• Groundedness detection (Preview): Detect ungrounded completions relative to sources; helps provenance and QA; preview scope limits.
• Protected material detection (text GA, code preview): Identifies copyrighted segments in completions; supports compliance; English scope.
• Blocklists and custom categories: Enforce organization-specific terms and custom detection; stored encrypted at rest with CMK options.

Minimal working example

Text moderation (Python) Requirements: requests or SDK; endpoint; subscription key or Entra auth.

import requests, json
endpoint = "https://YOUR-CONTENT-SAFETY.cognitiveservices.azure.com"
headers = {"Ocp-Apim-Subscription-Key": "KEY","Content-Type":"application/json"}
payload = {"text": "Sample text", "blocklistNames": []}
r = requests.post(f"{endpoint}/contentsafety/text:analyze?api-version=2024-09-01",
                  headers=headers, data=json.dumps(payload))
print(r.json())

Test: Inspect categoriesAnalysis severities; adjust thresholds; add blocklists.

Checklist

• Configure separate input/output filters; turn on Prompt Shields for inputs; add groundedness and protected material for outputs.
• Confirm API rate limits and payload size limits per modality/tier.
• Use blocklists for brand/PII terms; understand storage and deletion controls.

Where this doesn't apply

Offline moderation pipelines with proprietary classifiers; integrate but consider Content Safety for complementary coverage.

Sources

• What is Azure AI Content Safety? (GA) — https://learn.microsoft.com/en-us/azure/ai-services/content-safety/overview (accessed 2026-10-13).
• Documentation hub (GA) — https://learn.microsoft.com/en-us/azure/ai-services/content-safety/ (accessed 2026-10-13).
• Data, privacy, and security (GA) — https://learn.microsoft.com/en-us/azure/foundry/responsible-ai/content-safety/data-privacy (accessed 2026-10-13).

---