Annotation Guidelines

Annotation guidelines are the comprehensive, living documents that dictate exactly how human evaluators should rate, classify, or generate data for a specific AI training project. In the context of large language models, these guidelines serve as the source of truth for resolving edge cases and subjective disagreements.

A robust set of annotation guidelines will cover the primary evaluation criteria—such as helpfulness, honesty, and harmlessness. Crucially, it will provide concrete examples of how to penalize specific model behaviors. For instance, the guidelines might dictate that a hallucinated fact results in an automatic 'fail', while a minor formatting error only reduces the score by one point. They also define how to handle complex scenarios, such as when a user prompt contains a false premise, or when the user asks for a subjective opinion.

Because AI model behavior shifts rapidly, annotation guidelines are frequently updated. A rule that made sense in version 1 of a project might become obsolete if the model suddenly develops a tendency to overly apologize or refuse safe requests.

The strictness of these guidelines is what separates professional AI alignment from random crowdsourcing. The goal is to calibrate hundreds of human raters so that they make identical decisions when faced with the same ambiguous model output, producing a clean, consistent signal for the reward model.