Fact-Checking for LLM Evaluation

Fact-checking in LLM evaluation means tracing each factual claim a model makes back to a verifiable source, such as provided context, a citation, or an external ground truth, rather than judging whether the claim merely sounds plausible. It is the concrete, source-tracing sibling of groundedness, which asks a broader question about whether an output is supported by context at all.

This task is central to catching hallucination. A fluent, confident sentence with a fabricated statistic or an invented citation will pass a casual read but fail fact-checking the moment someone tries to verify it. Annotators doing this work are typically asked to label each claim individually as supported, unsupported, or unverifiable, rather than giving one holistic judgment on the whole response.

Fact-checking work has grown alongside retrieval-augmented systems, since RAG pipelines are only as trustworthy as their weakest unverified claim. A team can build a strong retrieval system and still ship an untrustworthy product if nobody checks whether the generated answer stayed faithful to what was retrieved.