Skip to main content
AITM · Module 1.3
M1.3

AITM-ECI — AI Experimentation Associate

18 articles · Stages: Calibrate

Articles in this module

  1. 1
    What an AI Experiment Is
    APP · M1.3-Art01
  2. 2
    Hypothesis Formulation and Metric Design
    APP · M1.3-Art02
  3. 3
    Offline Evaluation
    APP · M1.3-Art03
  4. 4
    Online Evaluation
    APP · M1.3-Art04
  5. 5
    Hyperparameter Search and Model Selection
    APP · M1.3-Art05
  6. 6
    Experiment Tracking, Reproducibility, and Replicability
    APP · M1.3-Art06
  7. 7
    Pipelines and Orchestration
    APP · M1.3-Art07
  8. 8
    Continuous Integration for ML
    APP · M1.3-Art08
  9. 9
    Continuous Delivery and Governed Promotion
    APP · M1.3-Art09
  10. 10
    Evaluating LLMs
    APP · M1.3-Art10
  11. 11
    Red-Team Experimentation for Safety
    APP · M1.3-Art11
  12. 12
    Experiment Cost and Compute Budget
    APP · M1.3-Art12
  13. 13
    Regulatory Documentation for Experiments
    APP · M1.3-Art13
  14. 14
    Experiment Brief and Experiment Report
    APP · M1.3-Art14
  15. 51
    Lab 01: Design and Execute an Offline Evaluation Harness
    APP · M1.3-Art51
  16. 52
    Lab 02: Design an Online A/B Test with Sample-Size Calculation and Rollback Criteria
    APP · M1.3-Art52
  17. 61
    Case Study: Zillow Offers and the Missing Shadow Evaluation
    APP · M1.3-Art61
  18. 71
    Artifact Template: Experiment Brief
    APP · M1.3-Art71