Skip to main content
COMPEL Body of Knowledge
Get Certified Platform

COMPEL Glossary / golden-dataset

Golden dataset

A versioned, labeled, license-cleared evaluation dataset used as the benchmark reference for an AI feature.

What this means in practice

Distinguished from production or training data by explicit curation, inter-annotator agreement review, and immutability-with-version. The golden dataset is the single artefact against which regression, release, and drift decisions are made.

Synonyms

golden set , reference evaluation dataset , eval gold set

Related Terms Network

This diagram shows how Golden dataset connects to other concepts in the COMPEL body of knowledge. Each linked node is a term you can explore in turn.

Related terms for Golden dataset Network graph showing 4 related glossary terms connected to Golden dataset. Golden dataset Evaluation… LLM-as-jud… Benchmark … Position b…
4 related concepts connected to Golden dataset.

The COMPEL Glossary Graph visualizes relationships between framework terminology, showing how concepts interconnect across domains, stages, and pillars. Term nodes cluster by pillar affiliation while cross-references reveal semantic dependencies — for example, how risk appetite connects to control effectiveness, model governance, and assurance requirements. This network representation helps practitioners navigate the framework vocabulary and understand that COMPEL terminology forms a coherent conceptual system rather than isolated definitions.

Accessible list of related terms
  • Evaluation harness — The infrastructure that runs capability, regression, safety, and human-review evaluations on an LLM feature on a defined cadence.
  • LLM-as-judge — An evaluation technique using a large language model to score outputs from another LLM on quality dimensions — helpfulness, correctness, safety — scaling evaluation beyond human-rater capacity.
  • Benchmark contamination — The presence of benchmark test data in foundation-model training corpora — whether through web crawling or deliberate inclusion — inflating reported benchmark scores and breaking the comparability of benchmark results across models.
  • Position bias (judge) — The systematic tendency of an LLM-as-judge to favour responses in a particular position (first, second, or last) when comparing candidates — independent of content quality.

See also

  • Evaluation harness — The infrastructure that runs capability, regression, safety, and human-review evaluations on an LLM feature on a defined cadence.
  • LLM-as-judge — An evaluation technique using a large language model to score outputs from another LLM on quality dimensions — helpfulness, correctness, safety — scaling evaluation beyond human-rater capacity.
  • Benchmark contamination — The presence of benchmark test data in foundation-model training corpora — whether through web crawling or deliberate inclusion — inflating reported benchmark scores and breaking the comparability of benchmark results across models.
  • Position bias (judge) — The systematic tendency of an LLM-as-judge to favour responses in a particular position (first, second, or last) when comparing candidates — independent of content quality.