Red Teaming (AI)

Red Teaming in AI is the adversarial practice of systematically probing a language model to discover harmful, unsafe, biased, or policy-violating outputs before the model is deployed to the public. Borrowed from cybersecurity, where red teams attack systems to find vulnerabilities, AI red teaming involves deliberately crafting complex prompts to bypass the model's safety filters.

Red teaming covers several domains. **Safety red teaming** attempts to trick the model into generating hate speech, providing instructions for illegal acts, or producing explicit content. **Capability red teaming** looks for specific skill failures, such as confidently answering medical queries incorrectly or hallucinating legal precedents. **Security red teaming** searches for prompt injection vulnerabilities or data exfiltration risks in agentic systems.

Effective red teaming requires deep creativity and an understanding of how models process language. Attackers use techniques like hypothetical role-play ('You are a fictional villain...'), complex encoding (asking for dangerous code in Base64), or overwhelming the model's context window with distractor text to slip malicious instructions past the safety classifiers.

Thorough documentation is just as important as the attack itself. Red teamers must record exactly which prompts succeeded, the model's failure mode, and the underlying vulnerability, allowing AI developers to patch the safety training datasets.