Fact-Checking for LLM Evaluation

Fact-checking in LLM evaluation means tracing each factual claim a model makes back to a verifiable source — provided context, a citation, or external ground truth — rather than judging whether the claim merely sounds plausible. It's the concrete, source-tracing sibling of groundedness, which asks a broader question about whether output is supported by context at all.

This task is central to catching hallucination: a fluent, confident sentence with a fabricated statistic or invented citation will pass a casual read but fail fact-checking the moment someone tries to verify it. Annotators doing this work are typically asked to label each claim individually (supported / unsupported / unverifiable) rather than giving one holistic judgment on the whole response.

Fact-checking work has grown alongside retrieval-augmented systems, since RAG pipelines are only as trustworthy as their weakest unverified claim.