Response Safety Grading
Scoring model outputs across defined safety risk dimensions.
What this means for trainers
Grade severity, not just presence — a response that's marginally risky and one that's severely dangerous should land in clearly different tiers, even though both technically involve some risk.
Response safety grading assigns a score or tier to a model output based on how much risk it poses across defined dimensions, such as physical harm, psychological harm, and legal risk, rather than a single pass or fail safety judgment. This granularity lets teams distinguish between a mildly risky response and a severely dangerous one, which matters for both training signal quality and for setting deployment thresholds.
This differs from toxicity annotation or policy enforcement in scope. Those focus on content categories, asking whether a response is abusive or violates a particular policy, while safety grading focuses on the severity of the consequence across whichever category applies.
The resulting scores feed into harmlessness training objectives alongside helpfulness signals, since a genuinely well-aligned model needs both. Being safe alone is not the goal if it comes at the cost of a response that is technically harmless but practically useless.
Related terms
Put this into practice
Browse open red-teaming, safety evaluation, and model-alignment roles from Alignerr, Mercor, Outlier, and more.
Browse AI training jobs