Rubric-Based Evaluation

Rubric-based evaluation breaks a holistic quality judgment into explicit, separately scored dimensions rather than asking for one overall rating. A response might be scored independently on correctness, helpfulness, safety, and tone, with each dimension defined clearly enough that different annotators land on similar scores for the same output.

This is the general framework underneath many specific evaluation types covered elsewhere in this glossary — summarization evaluation, code correctness evaluation, and instruction-following evaluation are all rubric-based evaluation applied to a specific domain with its own dimensions.

Well-designed rubrics anchor each score level with concrete examples rather than abstract descriptions ("a 3 looks like this" rather than "a 3 is moderately good"), since vague anchors are one of the fastest ways to introduce rubric drift across a large annotator pool.