Skip to main content

COMPEL Glossary / hypothesis

Hypothesis

A falsifiable statement an AI experiment is designed to support or reject — specifying an intervention, an expected direction of effect, a primary metric, and an acceptable effect size.

What this means in practice

Hypothesis-first experimentation (declared before running the experiment) prevents p-hacking, selective reporting, and post-hoc narrative fitting.

Synonyms

experiment hypothesis , testable hypothesis

See also

  • Primary metric — The single metric whose movement drives an experiment's ship/no-ship decision.
  • Guardrail metric — A secondary metric that must not degrade — typically beyond a declared tolerance — even if the primary metric improves.
  • AI experiment — A structured comparison producing evidence for a decision — about a model version, a prompt, a feature set, a retrieval strategy, or a deployment change.

Related articles in the Body of Knowledge